[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 499691312 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003109] x2apic enabled [ 0.005005] Switched APIC routing to physical x2apic. [ 0.006000] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008021] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009015] pid_max: default: 32768 minimum: 301 [ 0.011065] LSM: Security Framework initializing [ 0.012070] Yama: becoming mindful. [ 0.014012] SELinux: Initializing. [ 0.015000] *** VALIDATE selinux *** [ 0.023352] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028138] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029144] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030108] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032101] *** VALIDATE tmpfs *** [ 0.033438] *** VALIDATE proc *** [ 0.034239] *** VALIDATE cgroup *** [ 0.035009] *** VALIDATE cgroup2 *** [ 0.036246] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038073] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040028] Spectre V2 : User space: Vulnerable [ 0.041010] Speculative Store Bypass: Vulnerable [ 0.043793] debug: unmapping init [mem 0xffffffffb9059000-0xffffffffb9060fff] [ 0.045186] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046708] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047022] ... version: 2 [ 0.048013] ... bit width: 48 [ 0.049011] ... generic registers: 4 [ 0.050008] ... value mask: 0000ffffffffffff [ 0.051018] ... max period: 00007fffffffffff [ 0.052011] ... fixed-purpose events: 3 [ 0.053012] ... event mask: 000000070000000f [ 0.054299] rcu: Hierarchical SRCU implementation. [ 0.056435] smp: Bringing up secondary CPUs ... [ 0.057582] x86: Booting SMP configuration: [ 0.058024] .... node #0, CPUs: #1 #2 #3 [ 0.062012] smp: Brought up 1 node, 4 CPUs [ 0.064018] smpboot: Max logical packages: 1 [ 0.065012] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.147205] node 0 deferred pages initialised in 79ms [ 0.150421] devtmpfs: initialized [ 0.152261] x86/mm: Memory block size: 128MB [ 0.154070] gcov: version magic: 0x41383552 [ 0.157318] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.162113] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.165305] pinctrl core: initialized pinctrl subsystem [ 0.168155] [ 0.168665] ************************************************************* [ 0.171010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.174011] ** ** [ 0.178012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.180011] ** ** [ 0.183013] ** This means that this kernel is built to expose internal ** [ 0.186011] ** IOMMU data structures, which may compromise security on ** [ 0.189014] ** your system. ** [ 0.192013] ** ** [ 0.195013] ** If you see this message and you are not debugging the ** [ 0.198012] ** kernel, report this immediately to your vendor! ** [ 0.201013] ** ** [ 0.204014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.206012] ************************************************************* [ 0.209598] NET: Registered protocol family 16 [ 0.212409] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.217063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.220072] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.225062] cpuidle: using governor menu [ 0.226857] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.227418] PCI: Using configuration type 1 for base access [ 0.228179] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.237077] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.240040] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.246118] cryptd: max_cpu_qlen set to 1000 [ 0.249262] ACPI: Added _OSI(Module Device) [ 0.251015] ACPI: Added _OSI(Processor Device) [ 0.253014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.256012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.261332] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.268371] ACPI: Interpreter enabled [ 0.269059] ACPI: PM: (supports S0 S3 S4 S5) [ 0.271011] ACPI: Using IOAPIC for interrupt routing [ 0.273107] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.276428] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.287830] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.290039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.292017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.296073] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.301399] acpiphp: Slot [2] registered [ 0.303103] acpiphp: Slot [5] registered [ 0.305119] acpiphp: Slot [6] registered [ 0.306105] acpiphp: Slot [7] registered [ 0.308112] acpiphp: Slot [8] registered [ 0.309135] acpiphp: Slot [9] registered [ 0.311208] acpiphp: Slot [10] registered [ 0.312118] acpiphp: Slot [3] registered [ 0.314100] acpiphp: Slot [4] registered [ 0.315089] acpiphp: Slot [11] registered [ 0.317104] acpiphp: Slot [12] registered [ 0.318093] acpiphp: Slot [13] registered [ 0.320121] acpiphp: Slot [14] registered [ 0.321107] acpiphp: Slot [15] registered [ 0.323120] acpiphp: Slot [16] registered [ 0.324088] acpiphp: Slot [17] registered [ 0.326124] acpiphp: Slot [18] registered [ 0.327089] acpiphp: Slot [19] registered [ 0.329127] acpiphp: Slot [20] registered [ 0.331091] acpiphp: Slot [21] registered [ 0.332140] acpiphp: Slot [22] registered [ 0.334150] acpiphp: Slot [23] registered [ 0.336090] acpiphp: Slot [24] registered [ 0.337099] acpiphp: Slot [25] registered [ 0.339098] acpiphp: Slot [26] registered [ 0.340111] acpiphp: Slot [27] registered [ 0.341090] acpiphp: Slot [28] registered [ 0.343081] acpiphp: Slot [29] registered [ 0.344092] acpiphp: Slot [30] registered [ 0.346099] acpiphp: Slot [31] registered [ 0.347048] PCI host bridge to bus 0000:00 [ 0.348027] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.351022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.353040] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.355031] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.358022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.361020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.362143] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.364946] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.367903] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.384014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.388544] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.391017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.393016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.396059] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.398455] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.401926] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.404045] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.407828] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.412013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.426016] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.431019] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.436665] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.444014] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.456013] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.475017] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.487224] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.503015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.509013] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.533016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.546870] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.555016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.562014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.587015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.594272] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.602014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.611015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.632014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.644107] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.654014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.660012] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.684014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.694538] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.701013] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.710015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.730018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.741083] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.744390] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.747347] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.751392] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.754213] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.759260] iommu: Default domain type: Passthrough [ 0.761434] SCSI subsystem initialized [ 0.763101] ACPI: bus type USB registered [ 0.764187] usbcore: registered new interface driver usbfs [ 0.766058] usbcore: registered new interface driver hub [ 0.767056] usbcore: registered new device driver usb [ 0.768151] pps_core: LinuxPPS API ver. 1 registered [ 0.770007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.771039] PTP clock support registered [ 0.773059] EDAC MC: Ver: 3.0.0 [ 0.774528] PCI: Using ACPI for IRQ routing [ 0.776731] NetLabel: Initializing [ 0.777009] NetLabel: domain hash size = 128 [ 0.778012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.781073] NetLabel: unlabeled traffic allowed by default [ 0.784091] vgaarb: loaded [ 0.786176] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.787009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.792319] clocksource: Switched to clocksource kvm-clock [ 0.904081] VFS: Disk quotas dquot_6.6.0 [ 0.906122] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.909267] *** VALIDATE ramfs *** [ 0.910801] *** VALIDATE hugetlbfs *** [ 0.913039] pnp: PnP ACPI init [ 0.915511] pnp: PnP ACPI: found 6 devices [ 0.940407] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.943622] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.945183] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.947583] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.949957] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.952506] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.955301] NET: Registered protocol family 2 [ 0.957770] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.963759] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.968316] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.974384] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.978464] TCP: Hash tables configured (established 65536 bind 65536) [ 0.982118] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.985951] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.989307] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.993991] NET: Registered protocol family 1 [ 0.997048] RPC: Registered named UNIX socket transport module. [ 0.999370] RPC: Registered udp transport module. [ 1.000872] RPC: Registered tcp transport module. [ 1.002859] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.004865] NET: Registered protocol family 44 [ 1.006596] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.008705] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.010954] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.013355] PCI: CLS 0 bytes, default 64 [ 1.015471] Unpacking initramfs... [ 2.419573] debug: unmapping init [mem 0xffffa0713cc54000-0xffffa0713ffbffff] [ 2.424043] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.426604] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.429560] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.934848] Initialise system trusted keyrings [ 2.936850] Key type blacklist registered [ 2.939086] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.948256] zbud: loaded [ 2.951804] *** VALIDATE nfs *** [ 2.953152] *** VALIDATE nfs4 *** [ 2.954805] pstore: using deflate compression [ 2.958257] Platform Keyring initialized [ 3.065719] NET: Registered protocol family 38 [ 3.067534] Key type asymmetric registered [ 3.068886] Asymmetric key parser 'x509' registered [ 3.070780] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.073573] io scheduler mq-deadline registered [ 3.075219] io scheduler kyber registered [ 3.076747] io scheduler bfq registered [ 3.078194] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.080512] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.082810] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.085100] ACPI: Power Button [PWRF] [ 3.089504] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.094850] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.110571] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.123211] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.142970] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.176139] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.206063] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.211691] Non-volatile memory driver v1.3 [ 3.213583] Linux agpgart interface v0.103 [ 3.247936] virtio_blk virtio1: [vda] 146152 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.251142] vda: detected capacity change from 0 to 74829824 [ 3.268091] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.271091] vdb: detected capacity change from 0 to 1073741824 [ 3.287211] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.291565] vdc: detected capacity change from 0 to 2621440000 [ 3.313945] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.317720] vdd: detected capacity change from 0 to 2621440000 [ 3.339880] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.343463] vde: detected capacity change from 0 to 4294967296 [ 3.365140] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.368500] vdf: detected capacity change from 0 to 4294967296 [ 3.377248] libphy: Fixed MDIO Bus: probed [ 3.384644] usbcore: registered new interface driver usbserial_generic [ 3.386862] usbserial: USB Serial support registered for generic [ 3.389092] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.393131] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.394749] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.396915] mousedev: PS/2 mouse device common for all mice [ 3.400188] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.401480] rtc_cmos 00:05: RTC can wake from S4 [ 3.407564] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.407882] rtc_cmos 00:05: registered as rtc0 [ 3.413633] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.415210] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.416791] intel_pstate: CPU model not supported [ 3.424333] hid: raw HID events driver (C) Jiri Kosina [ 3.426615] usbcore: registered new interface driver usbhid [ 3.428843] usbhid: USB HID core driver [ 3.433640] drop_monitor: Initializing network drop monitor service [ 3.436251] Initializing XFRM netlink socket [ 3.438173] NET: Registered protocol family 10 [ 3.441300] Segment Routing with IPv6 [ 3.443142] NET: Registered protocol family 17 [ 3.445480] mpls_gso: MPLS GSO support [ 3.452084] RAS: Correctable Errors collector initialized. [ 3.454328] AVX version of gcm_enc/dec engaged. [ 3.456173] AES CTR mode by8 optimization enabled [ 3.539564] sched_clock: Marking stable (3539539425, 0)->(4429019607, -889480182) [ 3.543466] registered taskstats version 1 [ 3.545684] Loading compiled-in X.509 certificates [ 3.548470] zswap: loaded using pool lzo/zbud [ 3.573955] Key type big_key registered [ 3.586419] Key type encrypted registered [ 3.588051] ima: No TPM chip found, activating TPM-bypass! [ 3.590171] ima: Allocated hash algorithm: sha1 [ 3.591886] ima: No architecture policies found [ 3.593780] evm: Initialising EVM extended attributes: [ 3.595693] evm: security.selinux [ 3.596870] evm: security.ima [ 3.598125] evm: security.capability [ 3.599448] evm: HMAC attrs: 0x1 [ 3.603590] rtc_cmos 00:05: setting system clock to 2026-08-24 13:53:20 UTC (1787579600) [ 3.610226] debug: unmapping init [mem 0xffffffffba003000-0xffffffffba1fffff] [ 3.613736] debug: unmapping init [mem 0xffffffffb8d82000-0xffffffffb9058fff] [ 3.621050] Write protecting the kernel read-only data: 28672k [ 3.624244] debug: unmapping init [mem 0xffffffffb7403000-0xffffffffb75fffff] [ 3.626983] debug: unmapping init [mem 0xffffffffb7d14000-0xffffffffb7dfffff] [ 3.659350] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.668636] systemd[1]: Detected virtualization kvm. [ 3.670710] systemd[1]: Detected architecture x86-64. [ 3.672934] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.700633] systemd[1]: No hostname configured. [ 3.702422] systemd[1]: Set hostname to . [ 3.704647] random: systemd: uninitialized urandom read (16 bytes read) [ 3.707685] systemd[1]: Initializing machine ID from random generator. [ 3.758712] random: ln: uninitialized urandom read (6 bytes read) [ 3.854686] random: systemd: uninitialized urandom read (16 bytes read) [ 3.857642] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.862207] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.866437] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Swap. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.518191] device-mapper: uevent: version 1.0.3 [ 4.520848] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.278595] virtio_net virtio0 ens2: renamed from eth0 [ 5.299704] random: fast init done [ 5.403554] scsi host0: ata_piix [ 5.424247] scsi host1: ata_piix [ 5.425889] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.428057] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.135374] dracut-initqueue[579]: RTNETLINK answers: File exists [ 10.105964] random: crng init done [ 10.107620] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.595983] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.758733] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.007881] SELinux: Disabled at runtime. [ 12.072510] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.081125] systemd[1]: Detected virtualization kvm. [ 12.083105] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.605838] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.609249] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.617506] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.620667] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.623341] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.629825] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.638489] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target Paths. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. Starting udev Coldplug all Devices... [ 12.753759] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.138472] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.496662] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.546318] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.681569] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.699323] EDAC sbridge: Ver: 1.1.2 [ 15.310282] Key type dns_resolver registered [ 15.624332] NFS: Registering the id_resolver key type [ 15.625961] Key type id_resolver registered [ 15.627190] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg626-server login: [ 67.939155] libcfs: loading out-of-tree module taints kernel. [ 68.031280] Key type ._llcrypt registered [ 68.033902] Key type .llcrypt registered [ 68.203368] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 87.788490] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 89.708461] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 89.717465] alg: No test for adler32 (adler32-zlib) [ 91.393685] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 92.420333] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 94.243499] Key type lgssc registered [ 96.207850] Lustre: Echo OBD driver; http://www.lustre.org/ [ 114.371081] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 141.676163] hrtimer: interrupt took 2627657 ns [ 156.103618] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 167.366440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 167.402442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 168.611646] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 168.648260] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 168.743415] Lustre: lustre-MDT0000: new disk, initializing [ 168.888608] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 168.920660] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 173.342430] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.716749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 184.869637] Lustre: 6527:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 184.896641] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 184.905359] Lustre: Skipped 1 previous similar message [ 185.023166] Lustre: lustre-MDT0001: new disk, initializing [ 185.096772] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 185.124940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 185.135299] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 188.649932] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 193.762330] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 204.995178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 205.309617] Lustre: lustre-OST0000: new disk, initializing [ 205.313745] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 205.321045] Lustre: 8464:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 205.326593] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 205.415894] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 210.425612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 210.435249] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 210.492852] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 212.325742] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.688614] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.855338] Lustre: lustre-OST0001: new disk, initializing [ 225.861478] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.868727] Lustre: 9539:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.942247] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 232.429365] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.032824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 233.044860] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 233.101401] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 246.013635] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 252.306320] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 258.927838] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing check_logdir /tmp/testlogs/ [ 264.266638] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing yml_node [ 268.396191] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 270.835464] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 273.598646] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 275.473364] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 24 09:57:49 EDT 2026 [ 292.085964] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 293.801459] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 309.730056] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 309.738123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 309.739900] Lustre: Skipped 3 previous similar messages [ 309.753441] Lustre: Skipped 3 previous similar messages [ 314.859406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.873467] Lustre: Skipped 3 previous similar messages [ 316.091789] Lustre: server umount lustre-MDT0000 complete [ 319.892367] LustreError: 6517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787579916 with bad export cookie 2368508769700257123 [ 319.894723] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 319.907748] LustreError: 6517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 319.968906] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 319.997334] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 320.007636] Lustre: Skipped 1 previous similar message [ 325.097266] LustreError: 9206:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 325.098047] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 325.109675] LustreError: 9206:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 329.186530] LustreError: 6534:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 329.206333] LustreError: 6534:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 334.308880] LustreError: 9206:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 334.309134] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 334.310121] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? [ 334.373956] Lustre: Skipped 4 previous similar messages [ 334.674666] Lustre: server umount lustre-MDT0001 complete [ 338.685272] Lustre: server umount lustre-OST0000 complete [ 342.619448] Lustre: server umount lustre-OST0001 complete [ 349.792735] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 360.715941] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 397.635199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.909301] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 397.956240] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 398.036810] Lustre: lustre-MDT0000: new disk, initializing [ 398.116266] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 398.134629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 402.872643] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 414.263470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 414.391794] Lustre: 15093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 414.450270] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 414.453465] Lustre: Skipped 1 previous similar message [ 414.532486] Lustre: lustre-MDT0001: new disk, initializing [ 414.640626] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 414.672989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 414.684099] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 419.671789] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 429.524348] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 435.516432] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 442.727884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 443.099070] Lustre: lustre-OST0000: new disk, initializing [ 443.103637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 443.113771] Lustre: 17064:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 443.265268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 444.474490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 444.478520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 444.553954] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 449.463578] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 459.680549] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 464.267573] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 464.544755] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 469.617436] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 469.874594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 475.114149] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 475.131397] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 475.147543] Lustre: Skipped 1 previous similar message [ 475.154575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 475.621045] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 485.856149] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 486.058666] Lustre: server umount lustre-OST0000 complete [ 490.982475] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 490.996741] Lustre: Skipped 2 previous similar messages [ 496.806210] Lustre: server umount lustre-MDT0000 complete [ 500.476757] LustreError: 15900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580097 with bad export cookie 2368508769700258656 [ 500.482310] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 500.493264] LustreError: 15900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 501.078191] Lustre: server umount lustre-MDT0001 complete [ 506.484540] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:01:40 (1787580100) [ 512.925177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 513.303803] LustreError: 19343:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.329372] LustreError: 19343:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 513.388599] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 517.054234] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 518.631736] LustreError: 19344:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 525.057134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 529.771813] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 537.949788] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 543.895129] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 550.709690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 557.952921] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 566.007506] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 573.295305] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 581.097211] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 581.099269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 581.115256] Lustre: Skipped 1 previous similar message [ 581.122355] Lustre: Skipped 9 previous similar messages [ 585.560402] Lustre: server umount lustre-MDT0000 complete [ 586.215268] LustreError: 19344:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 586.238894] LustreError: 19344:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 589.403162] LustreError: 19324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580186 with bad export cookie 2368508769700259538 [ 589.408544] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 589.420319] LustreError: 19324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 589.827544] Lustre: server umount lustre-MDT0001 complete [ 604.170690] Lustre: server umount lustre-OST0000 complete [ 627.377328] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 640.286779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 640.902453] LustreError: 23724:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 641.032795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 641.037679] Lustre: Skipped 2 previous similar messages [ 646.598760] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 656.232738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 661.984552] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 664.975819] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 672.407227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.323536] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 688.186881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 688.353431] Lustre: lustre-OST0001: new disk, initializing [ 688.360525] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 688.372663] Lustre: 26041:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 690.268774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 690.292462] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 690.433856] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 695.299267] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 704.653770] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 708.352132] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 721.387449] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 721.397313] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 721.414336] Lustre: Skipped 3 previous similar messages [ 721.429279] Lustre: Skipped 3 previous similar messages [ 726.979293] Lustre: server umount lustre-MDT0000 complete [ 730.338464] LustreError: 23706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580327 with bad export cookie 2368508769700260385 [ 730.340280] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 730.364025] LustreError: 23706:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 730.801295] Lustre: server umount lustre-MDT0001 complete [ 745.191593] Lustre: server umount lustre-OST0000 complete [ 748.004418] Lustre: 3661:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787580328/real 1787580328] req@ffffa0719d00d880 x1874413160219136/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787580344 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 748.039062] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 748.057866] Lustre: Skipped 1 previous similar message [ 749.247675] Lustre: server umount lustre-OST0001 complete [ 766.735554] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 776.822189] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 777.227552] LustreError: 29061:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 777.248891] LustreError: 29061:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 777.322935] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 777.329631] Lustre: Skipped 3 previous similar messages [ 782.224425] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 790.214939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 795.004879] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 797.810562] Lustre: 30209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 804.485044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 811.603816] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 821.381789] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.551712] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 836.338873] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 840.032155] Lustre: 32089:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 846.817985] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 846.837349] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 852.471240] Lustre: server umount lustre-MDT0000 complete [ 853.479730] LustreError: 29797:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 853.499260] LustreError: 29797:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 856.162723] LustreError: 31399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580453 with bad export cookie 2368508769700261680 [ 856.172674] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 856.175363] LustreError: 31399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 856.567818] Lustre: server umount lustre-MDT0001 complete [ 871.194740] Lustre: server umount lustre-OST0000 complete [ 874.784135] Lustre: 3659:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787580455/real 1787580455] req@ffffa07083e29880 x1874413160269568/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787580471 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 875.460799] Lustre: server umount lustre-OST0001 complete [ 885.079962] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:07:59 (1787580479) [ 914.163354] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 924.453649] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 929.743840] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 938.996617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 944.616623] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 947.945619] Lustre: 36041:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 954.787082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 962.281503] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 971.492543] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 978.601719] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 987.122358] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 990.991371] Lustre: 37920:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 997.350501] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 997.367565] Lustre: Skipped 5 previous similar messages [ 997.367927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 997.383705] Lustre: Skipped 13 previous similar messages [ 1004.875909] Lustre: server umount lustre-MDT0000 complete [ 1007.593251] LustreError: 34887:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1007.620116] LustreError: 34887:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 1008.318856] LustreError: 37922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580605 with bad export cookie 2368508769700262786 [ 1008.322916] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1008.333286] LustreError: 37922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1008.683921] Lustre: server umount lustre-MDT0001 complete [ 1023.134640] Lustre: server umount lustre-OST0000 complete [ 1036.516643] Lustre: server umount lustre-OST0001 complete [ 1045.423181] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:10:39 (1787580639) [ 1061.152180] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1071.128718] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1071.528423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1071.533658] Lustre: Skipped 7 previous similar messages [ 1075.774833] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1084.426709] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1089.583689] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1092.678822] Lustre: 41214:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1099.673941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1105.820329] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1114.746250] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1121.083560] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1129.879443] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1133.934914] Lustre: 43092:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1134.709431] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1134.712970] Lustre: Skipped 1 previous similar message [ 1141.219979] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1141.232449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1141.257514] Lustre: Skipped 1 previous similar message [ 1144.379662] Lustre: server umount lustre-MDT0000 complete [ 1148.140942] LustreError: 40045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580745 with bad export cookie 2368508769700263892 [ 1148.145756] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1148.152950] LustreError: 40045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1154.543573] Lustre: server umount lustre-MDT0001 complete [ 1163.410042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1173.485148] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x20dea22d5961bc3b [ 1173.508634] Lustre: MGC192.168.206.126@tcp: Connection restored to 0@lo (at 0@lo) [ 1178.458302] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1183.160036] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1187.688967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1192.808351] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1193.466949] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1193.472882] Lustre: Skipped 1 previous similar message [ 1200.177448] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1201.886630] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1207.103213] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1208.798136] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1219.059526] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1220.571878] Lustre: server umount lustre-MDT0000 complete [ 1224.148043] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580821 with bad export cookie 2368508769700265019 [ 1224.151525] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1224.155299] LustreError: 40046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1224.531320] Lustre: server umount lustre-MDT0001 complete [ 1230.140463] Lustre: server umount lustre-OST0000 complete [ 1233.571915] Lustre: server umount lustre-OST0001 complete [ 1241.423967] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:13:55 (1787580835) [ 1256.387912] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1267.530383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1268.051916] LustreError: 47698:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1268.078490] LustreError: 47698:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 28 previous similar messages [ 1273.681323] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1282.612916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1287.879669] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1291.051690] Lustre: 48850:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1298.628759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1305.709992] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1315.642710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1322.346318] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1329.843554] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1334.337296] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1346.538186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1346.554551] Lustre: Skipped 23 previous similar messages [ 1348.265243] Lustre: server umount lustre-MDT0000 complete [ 1352.170372] LustreError: 50736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787580949 with bad export cookie 2368508769700265810 [ 1352.180963] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1352.188399] LustreError: 50736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1352.542355] Lustre: server umount lustre-MDT0001 complete [ 1366.241326] Lustre: server umount lustre-OST0000 complete [ 1380.246540] Lustre: server umount lustre-OST0001 complete [ 1390.475228] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:16:24 (1787580984) [ 1393.589521] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1395.916989] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:16:29 (1787580989) [ 1408.374805] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 1419.394864] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1447.366137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1447.513850] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1447.657689] Lustre: lustre-MDT0000: new disk, initializing [ 1447.842688] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1452.629314] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1463.778734] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1471.445773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1471.678266] Lustre: lustre-OST0000: new disk, initializing [ 1471.681366] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1471.684987] Lustre: Skipped 1 previous similar message [ 1471.694734] Lustre: 54929:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1472.880609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1472.897058] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1472.930719] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1477.932842] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1487.452048] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1490.215775] Lustre: DEBUG MARKER: create test files [ 1524.195232] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1524.206596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1524.225110] Lustre: Skipped 15 previous similar messages [ 1525.986335] Lustre: server umount lustre-OST0000 complete [ 1530.340193] Lustre: server umount lustre-MDT0000 complete [ 1544.722370] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 1548.058990] Key type lgssc unregistered [ 1548.405657] LNet: 56566:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1548.413174] LNetError: 56566:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1548.430218] LNet: Removed LNI 192.168.206.126@tcp [ 1549.571240] Key type .llcrypt unregistered [ 1549.572927] Key type ._llcrypt unregistered [ 1583.246945] Key type ._llcrypt registered [ 1583.250335] Key type .llcrypt registered [ 1583.462369] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1585.434201] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1585.450359] alg: No test for adler32 (adler32-zlib) [ 1586.644393] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 1586.983350] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 1588.747102] Key type lgssc registered [ 1589.794473] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1599.833641] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1599.889811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1601.570841] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1606.408564] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1615.759818] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1625.348785] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1626.175657] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1627.209986] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:100 to 0x240000400:129) [ 1627.330665] Lustre: 59305:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1627.361827] Lustre: 59305:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1627.388885] Lustre: 59305:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1627.407946] Lustre: 59305:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1627.424115] Lustre: 59305:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1627.448104] Lustre: 59305:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1634.636606] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1643.857604] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1649.256120] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1655.711874] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1662.947416] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1662.957644] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1662.972334] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1664.733426] Lustre: server umount lustre-OST0000 complete [ 1668.956892] Lustre: server umount lustre-MDT0000 complete [ 1680.626300] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 1684.168216] Key type lgssc unregistered [ 1684.506822] LNet: 60924:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1684.521214] LNetError: 60924:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1684.542279] LNet: Removed LNI 192.168.206.126@tcp [ 1685.382225] Key type .llcrypt unregistered [ 1685.387688] Key type ._llcrypt unregistered [ 1722.491412] Key type ._llcrypt registered [ 1722.495584] Key type .llcrypt registered [ 1722.617372] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1723.663133] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1723.706444] alg: No test for adler32 (adler32-zlib) [ 1724.874188] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 1725.050517] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 1726.704208] Key type lgssc registered [ 1728.003708] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1736.464304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1736.481364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1738.112960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1743.530810] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1752.832890] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1760.198475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1760.679378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1762.732212] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1762.816222] Lustre: 63761:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 1762.824663] Lustre: 63761:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 1762.829430] Lustre: 63761:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1762.840705] Lustre: 63761:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1762.848552] Lustre: 63761:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1762.854565] Lustre: 63761:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1767.642309] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1776.515790] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1782.570266] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1793.508204] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1793.543080] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1797.377519] Lustre: server umount lustre-OST0000 complete [ 1801.217265] Lustre: server umount lustre-MDT0000 complete [ 1813.710733] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 1816.558828] Key type lgssc unregistered [ 1816.913934] LNet: 65321:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1816.923959] LNetError: 65321:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1816.967543] LNet: Removed LNI 192.168.206.126@tcp [ 1817.884948] Key type .llcrypt unregistered [ 1817.887591] Key type ._llcrypt unregistered [ 1837.830826] Key type ._llcrypt registered [ 1837.834877] Key type .llcrypt registered [ 1837.985532] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 1856.309805] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 1858.664965] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1858.754934] alg: No test for adler32 (adler32-zlib) [ 1860.018556] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 1860.444764] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 1862.200750] Key type lgssc registered [ 1863.667580] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1902.518309] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1902.534724] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1903.799300] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1903.849194] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1903.917791] Lustre: lustre-MDT0000: new disk, initializing [ 1904.085632] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1904.117711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1909.389540] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1920.731943] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1920.837189] Lustre: 68476:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1920.918324] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1920.925886] Lustre: Skipped 1 previous similar message [ 1920.997354] Lustre: lustre-MDT0001: new disk, initializing [ 1921.083096] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1921.154728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1921.173273] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1925.135946] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1935.748370] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1942.937893] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1950.415783] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1950.663659] Lustre: lustre-OST0000: new disk, initializing [ 1950.670171] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1950.679580] Lustre: 70445:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1950.794142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1957.223782] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 1957.498037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1957.515378] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1957.659724] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1967.615724] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1973.080176] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1973.373429] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1978.544477] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1978.747964] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1983.457544] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1983.459102] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1983.470355] LustreError: Skipped 1 previous similar message [ 1983.477601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1983.478883] Lustre: Skipped 1 previous similar message [ 1983.485819] Lustre: Skipped 1 previous similar message [ 1986.570038] Lustre: server umount lustre-OST0000 complete [ 1992.674841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1992.688612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1992.715110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1993.704796] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1993.733698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1993.740325] Lustre: Skipped 1 previous similar message [ 1998.684281] Lustre: server umount lustre-MDT0000 complete [ 1998.819544] LustreError: 68486:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1998.831160] LustreError: 68486:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2003.071433] LustreError: 69272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787581599 with bad export cookie 15424401234094837300 [ 2003.075922] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2003.080727] LustreError: 69272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2003.664566] Lustre: server umount lustre-MDT0001 complete [ 2012.745113] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:26:47 (1787581607) [ 2024.229376] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 2070.037422] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 2080.774623] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 2118.084038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2118.286953] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2118.315958] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2118.379443] Lustre: lustre-MDT0000: new disk, initializing [ 2118.466496] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2118.486457] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2123.449927] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2134.924734] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2135.054938] Lustre: 76468:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2135.114469] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2135.117924] Lustre: Skipped 1 previous similar message [ 2135.180354] Lustre: lustre-MDT0001: new disk, initializing [ 2135.273995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2135.310598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2135.323035] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2139.976465] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2150.075104] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2156.213378] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2163.868364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2164.153686] Lustre: lustre-OST0000: new disk, initializing [ 2164.156901] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2164.163296] Lustre: 78436:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2164.224206] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2170.439062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2170.457904] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2170.610802] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2170.820766] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2180.254781] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2185.130164] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2185.385462] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2189.828088] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2190.050632] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2196.451817] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2196.470339] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2196.482090] Lustre: Skipped 1 previous similar message [ 2196.482946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2196.494883] Lustre: Skipped 1 previous similar message [ 2198.710286] Lustre: server umount lustre-OST0000 complete [ 2206.698832] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2206.717571] Lustre: Skipped 1 previous similar message [ 2206.724735] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2206.740135] Lustre: Skipped 1 previous similar message [ 2209.049261] Lustre: server umount lustre-MDT0000 complete [ 2211.817765] LustreError: 76480:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2211.833627] LustreError: 76480:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2213.141412] LustreError: 76461:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787581810 with bad export cookie 15424401234094838175 [ 2213.144905] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2213.163350] LustreError: 76461:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2213.715468] Lustre: server umount lustre-MDT0001 complete [ 2222.949050] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:30:17 (1787581817) [ 2230.236524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2230.663812] LustreError: 80849:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2230.735200] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2235.497404] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2235.875612] LustreError: 80850:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2240.997933] LustreError: 80849:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2244.253227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2249.498399] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2258.010754] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2264.268880] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2272.303500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2272.731061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2272.738094] Lustre: Skipped 1 previous similar message [ 2278.352388] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2288.055306] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2293.418969] Lustre: 83521:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 9b064996-c13f-47ab-8179-e8e5a16af33b at adminstrative request [ 2295.276478] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2295.300375] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2305.527750] Lustre: MGS: Client 9b064996-c13f-47ab-8179-e8e5a16af33b (at 0@lo) reconnecting [ 2305.545278] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd60e7b334f51d518 to 0xd60e7b334f51d7b8 [ 2305.602844] Lustre: MGC192.168.206.126@tcp: Connection restored to 0@lo (at 0@lo) [ 2331.364560] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2331.516896] Lustre: lustre-OST0001: new disk, initializing [ 2331.521969] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2331.528448] Lustre: 83915:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2338.129733] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2338.436994] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2338.451283] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2338.628184] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2348.940522] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2354.168018] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2354.169532] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2354.177313] LustreError: Skipped 1 previous similar message [ 2354.179412] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2354.218470] Lustre: Skipped 1 previous similar message [ 2358.757849] Lustre: server umount lustre-OST0001 complete [ 2359.283336] LustreError: 82671:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2359.300212] LustreError: 82671:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2362.849879] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2362.865083] LustreError: Skipped 1 previous similar message [ 2368.804831] Lustre: server umount lustre-OST0000 complete [ 2372.577124] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2372.581471] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2372.595654] Lustre: Skipped 2 previous similar messages [ 2372.602486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2372.606616] Lustre: Skipped 4 previous similar messages [ 2386.400735] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2386.550653] Lustre: server umount lustre-MDT0000 complete [ 2389.990389] LustreError: 81841:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2390.005713] LustreError: 81841:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2390.355065] LustreError: 80831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787581987 with bad export cookie 15424401234094839736 [ 2390.369209] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2390.374251] LustreError: 80831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2390.830765] Lustre: server umount lustre-MDT0001 complete [ 2400.715249] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:33:14 (1787581994) [ 2402.736844] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2404.570816] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:33:18 (1787581998) [ 2406.264445] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2408.178953] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:33:22 (1787582002) [ 2409.993993] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2412.736522] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:33:26 (1787582006) [ 2414.855937] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2417.302765] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:33:31 (1787582011) [ 2419.381751] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2425.966355] loop: module loaded [ 2425.983379] loop0: detected capacity change from 0 to 4294967296 [ 2426.065107] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2428.001866] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2431.995191] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2435.681680] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2465.674559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2465.740935] LustreError: 88042:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2465.757281] LustreError: 88042:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2465.772556] LustreError: 88042:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2465.788393] LustreError: 88042:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2489.107369] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 2500.650751] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 2544.228985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2544.579585] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2544.610606] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2544.696808] Lustre: lustre-MDT0000: new disk, initializing [ 2544.818145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2544.827712] Lustre: Skipped 1 previous similar message [ 2544.868267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2551.114868] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2563.715365] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2563.860899] Lustre: 90907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2564.132923] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2568.795247] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2579.113613] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2585.645326] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2592.600210] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2593.080660] Lustre: 92874:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2599.832989] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2603.098960] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2603.281946] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2610.217729] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2615.740310] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2616.022976] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2621.731206] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2621.988906] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2629.091051] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2629.096413] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2629.105925] LustreError: Skipped 1 previous similar message [ 2629.106486] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2629.106489] Lustre: Skipped 6 previous similar messages [ 2629.145443] Lustre: Skipped 2 previous similar messages [ 2630.509738] Lustre: server umount lustre-OST0000 complete [ 2641.408556] Lustre: server umount lustre-MDT0000 complete [ 2644.460700] LustreError: 90917:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2644.485343] LustreError: 90917:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2645.495376] LustreError: 90900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582242 with bad export cookie 15424401234094840352 [ 2645.513506] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2645.517386] LustreError: 90900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2645.994710] Lustre: server umount lustre-MDT0001 complete [ 2656.433155] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:37:30 (1787582250) [ 2663.839876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2668.948376] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2678.052388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2678.530455] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2678.537433] Lustre: Skipped 3 previous similar messages [ 2683.370878] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2690.915924] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2695.809295] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2703.291915] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2710.364361] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2717.642789] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2721.806094] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2721.964539] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2725.436624] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2725.607858] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2732.884632] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2733.022152] Lustre: lustre-OST0001: new disk, initializing [ 2733.024861] Lustre: Skipped 2 previous similar messages [ 2733.027596] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2733.030899] Lustre: Skipped 3 previous similar messages [ 2733.034827] Lustre: 98658:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2740.443408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2740.464335] Lustre: Skipped 2 previous similar messages [ 2740.471960] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2740.617108] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2741.485453] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2750.921304] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2756.848938] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2757.079986] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2762.062502] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2762.351316] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2783.392783] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2812.985958] Lustre: Failing over lustre-MDT0000 [ 2815.458738] Lustre: server umount lustre-MDT0000 complete [ 2815.963868] LustreError: 95283:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2815.982099] LustreError: 95283:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2816.998632] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2817.013296] LustreError: Skipped 1 previous similar message [ 2817.017047] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2817.033755] Lustre: Skipped 2 previous similar messages [ 2833.889037] Lustre: 66698:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787582414/real 1787582414] req@000000004965c671 x1874415014135296/t0(0) o400->MGC192.168.206.126@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787582430 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2833.935608] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2841.587359] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2841.589968] LDISKFS-fs (dm-0): recovery complete [ 2841.596774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2843.108932] LustreError: 66696:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000215b9a64 x1874415014143616/t0(0) o250->MGC192.168.206.126@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2843.390027] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2846.642445] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2846.650064] Lustre: 95284:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2848.743771] Lustre: 95288:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2848.758512] Lustre: 95288:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2848.781885] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 2848.902371] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2848.949635] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 2849.237203] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2849.589612] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2849.601893] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2849.944093] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2849.954395] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2849.972111] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2850.016048] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2850.677962] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2850.710848] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2851.040133] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2851.047692] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2852.042532] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2852.056883] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2852.699938] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2852.719947] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2853.056308] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2853.058679] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2856.237875] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2856.276749] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2856.927600] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2856.958891] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2857.288225] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2857.297716] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2864.241132] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2864.248955] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2865.210554] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2865.218337] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2865.536458] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2865.540130] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 25 previous similar messages [ 2880.434418] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2880.447168] Lustre: 101514:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2881.393440] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2881.410965] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2881.745062] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2881.756845] LustreError: 101514:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2906.112148] Lustre: 101514:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2906.125752] Lustre: 101514:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2906.143320] Lustre: 101514:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ab786d99-1a16-4417-b1c6-b557eb12c8c7@192.168.206.26@tcp [ 2906.159160] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2906.169736] Lustre: 101514:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000dfda5d0c x1874414992954880/t0(8589934776) o35->ab786d99-1a16-4417-b1c6-b557eb12c8c7@192.168.206.26@tcp:723/0 lens 392/0 e 0 to 0 dl 1787582513 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2906.196674] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2906.215439] Lustre: lustre-MDT0000: Denying connection for new client ab786d99-1a16-4417-b1c6-b557eb12c8c7 (at 192.168.206.26@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2906.227698] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2906.300541] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2906.309889] Lustre: Skipped 2 previous similar messages [ 2906.333966] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2906.366745] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2906.370315] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2918.881882] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2918.892099] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2918.901811] Lustre: Skipped 4 previous similar messages [ 2924.837296] Lustre: server umount lustre-OST0000 complete [ 2935.727941] Lustre: server umount lustre-OST0001 complete [ 2945.936784] Lustre: server umount lustre-MDT0000 complete [ 2946.019927] LustreError: 95285:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2946.038640] LustreError: 95285:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 44 previous similar messages [ 2949.418256] LustreError: 95270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582546 with bad export cookie 15424401234094891564 [ 2949.420030] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2949.436465] LustreError: 95270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2949.922252] Lustre: server umount lustre-MDT0001 complete [ 2959.792713] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:42:33 (1787582553) [ 2969.835956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2969.851222] Lustre: *** cfs_fail_loc=197, val=0*** [ 2985.761068] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2990.883329] LustreError: 103490:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.126@tcp: failed processing log, type 4: rc = -110 [ 2996.021559] Lustre: *** cfs_fail_loc=197, val=0*** [ 2996.025531] Lustre: Skipped 65 previous similar messages [ 3016.609172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3016.622862] Lustre: Skipped 3 previous similar messages [ 3022.240949] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3028.599961] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3030.841995] Lustre: server umount lustre-OST0000 complete [ 3039.313259] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 10:43:53 (1787582633) [ 3061.221818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3066.394676] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3076.774622] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3081.159913] LustreError: 105144:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3081.172606] LustreError: 105144:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3081.180347] LustreError: 105144:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3081.595350] Lustre: server umount lustre-MDT0000 complete [ 3113.515416] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 3124.398544] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 3166.144800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3166.334028] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3166.354402] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3166.437546] Lustre: lustre-MDT0000: new disk, initializing [ 3166.528682] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3170.718945] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3181.391431] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3181.574481] Lustre: 109321:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3181.830090] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3186.236820] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3195.616818] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3201.450386] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3208.395107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3208.699456] Lustre: 111288:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3215.425581] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3218.634377] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3225.561734] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3229.990563] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3230.216656] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3234.364492] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3234.581368] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3239.393147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3239.394324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3239.412881] LustreError: Skipped 2 previous similar messages [ 3239.416124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3239.416131] Lustre: Skipped 6 previous similar messages [ 3239.494186] Lustre: Skipped 10 previous similar messages [ 3242.748111] Lustre: server umount lustre-OST0000 complete [ 3253.682613] Lustre: server umount lustre-MDT0000 complete [ 3254.753336] LustreError: 109331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3254.762725] LustreError: 109331:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3258.681538] LustreError: 109312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787582855 with bad export cookie 15424401234094902204 [ 3258.690483] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3258.719260] LustreError: 109312:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3259.342533] Lustre: server umount lustre-MDT0001 complete [ 3270.012731] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:47:44 (1787582864) [ 3279.276904] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 3282.080890] Key type lgssc unregistered [ 3282.407520] LNet: 113778:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3282.419593] LNetError: 113778:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3282.438769] LNet: Removed LNI 192.168.206.126@tcp [ 3283.291572] Key type .llcrypt unregistered [ 3283.299594] Key type ._llcrypt unregistered [ 3302.216557] Key type ._llcrypt registered [ 3302.222911] Key type .llcrypt registered [ 3302.330303] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 3318.109375] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 3319.086496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3319.169052] alg: No test for adler32 (adler32-zlib) [ 3320.274502] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 3320.493163] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 3322.168160] Key type lgssc registered [ 3323.399535] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3369.913299] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3369.940665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3371.197477] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3371.227697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3371.334836] Lustre: lustre-MDT0000: new disk, initializing [ 3371.449576] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3371.466835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3375.829291] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3386.055743] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3386.373625] Lustre: lustre-OST0000: new disk, initializing [ 3386.377465] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3386.387451] Lustre: Skipped 1 previous similar message [ 3386.392417] Lustre: 117955:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3386.477735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3389.045300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3389.056394] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3389.080884] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3392.978120] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3403.588754] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3403.715620] Lustre: lustre-OST0001: new disk, initializing [ 3403.719486] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3403.724698] Lustre: 118819:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3403.783254] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3409.707510] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3411.107261] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3411.113935] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3411.144270] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3421.668676] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3421.674199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3421.696988] Lustre: Skipped 1 previous similar message [ 3424.347860] Lustre: server umount lustre-MDT0000 complete [ 3430.540034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3435.034172] LustreError: 117141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583031 with bad export cookie 10167566963359514724 [ 3435.045566] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3435.130948] Lustre: server umount lustre-OST0000 complete [ 3438.908450] Lustre: server umount lustre-OST0001 complete [ 3447.450471] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:50:41 (1787583041) [ 3469.531782] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:51:03 (1787583063) [ 3479.448452] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 3504.765524] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:51:38 (1787583098) [ 3539.799915] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 3549.501311] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 3582.923962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.306431] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3583.386472] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3583.534531] Lustre: lustre-MDT0000: new disk, initializing [ 3583.643687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3583.666382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3589.445852] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3600.988473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3601.109505] Lustre: 126570:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3601.151124] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3601.154429] Lustre: Skipped 1 previous similar message [ 3601.208464] Lustre: lustre-MDT0001: new disk, initializing [ 3601.273174] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3601.319360] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3601.334587] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3605.609206] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3614.593544] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3621.708709] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3629.612251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3630.035520] Lustre: lustre-OST0000: new disk, initializing [ 3630.038264] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3630.043311] Lustre: 128541:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3630.122279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3636.338483] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3637.840294] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3637.850525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3638.020425] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3645.514356] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3649.786845] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3650.049857] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3654.133643] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3654.320417] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3658.729319] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3658.731476] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3658.734229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3658.734799] Lustre: Skipped 1 previous similar message [ 3658.776666] Lustre: Skipped 1 previous similar message [ 3662.266545] Lustre: server umount lustre-OST0000 complete [ 3667.939491] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3667.945954] LustreError: Skipped 1 previous similar message [ 3667.951146] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3667.971720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3667.983280] Lustre: Skipped 1 previous similar message [ 3674.081646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3674.086086] Lustre: Skipped 3 previous similar messages [ 3678.180480] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3678.199662] Lustre: Skipped 1 previous similar message [ 3680.224136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3680.402750] Lustre: server umount lustre-MDT0000 complete [ 3684.303513] LustreError: 126562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583281 with bad export cookie 10167566963359515949 [ 3684.312751] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3684.323029] LustreError: 126562:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3684.767328] Lustre: server umount lustre-MDT0001 complete [ 3694.001993] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:54:48 (1787583288) [ 3700.976969] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3701.464747] LustreError: 130955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3701.488848] LustreError: 130955:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3701.529878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3706.232723] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3706.851384] LustreError: 130956:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3711.974823] LustreError: 130955:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3715.074049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3719.781744] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3726.661589] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3733.360821] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3742.484063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3742.949563] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3742.959848] Lustre: Skipped 1 previous similar message [ 3751.327792] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3758.837931] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3766.734288] Lustre: *** cfs_fail_loc=159, val=0*** [ 3783.131347] Lustre: lustre-MDT0000: Client c8b1596c-ad0a-48b2-b86d-57bf62f82a66 (at 192.168.206.26@tcp) reconnecting [ 3784.899232] Lustre: *** cfs_fail_loc=159, val=0*** [ 3784.900947] Lustre: Skipped 5 previous similar messages [ 3800.578799] Lustre: lustre-MDT0000: Client c8b1596c-ad0a-48b2-b86d-57bf62f82a66 (at 192.168.206.26@tcp) reconnecting [ 3818.977936] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3818.987362] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3818.998759] Lustre: Skipped 1 previous similar message [ 3819.007322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3822.049971] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3824.356176] Lustre: server umount lustre-OST0000 complete [ 3828.198221] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3828.214478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3828.231526] Lustre: Skipped 1 previous similar message [ 3834.396957] Lustre: server umount lustre-MDT0000 complete [ 3837.411705] LustreError: 130951:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3837.441130] LustreError: 130951:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3838.138560] LustreError: 130935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583435 with bad export cookie 10167566963359516831 [ 3838.146759] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3838.161200] LustreError: 130935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3838.468070] Lustre: server umount lustre-MDT0001 complete [ 3850.834989] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 3854.096674] Key type lgssc unregistered [ 3854.513875] LNet: 134737:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3854.524356] LNetError: 134737:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3854.550258] LNet: Removed LNI 192.168.206.126@tcp [ 3855.466868] Key type .llcrypt unregistered [ 3855.469271] Key type ._llcrypt unregistered [ 3880.716734] Key type ._llcrypt registered [ 3880.719182] Key type .llcrypt registered [ 3880.816625] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 3881.584101] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3881.594669] alg: No test for adler32 (adler32-zlib) [ 3882.594246] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 3882.868601] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 3884.616131] Key type lgssc registered [ 3885.952956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3895.015084] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3895.045843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3896.774240] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3901.722614] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3907.050344] LustreError: 136155:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3907.070603] LustreError: 136155:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3910.904790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3911.304224] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3915.316636] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3922.880374] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3929.213718] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3936.011984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3936.232677] LustreError: 137975:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3936.409672] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3941.569065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3944.527933] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 3952.001604] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3960.975406] Lustre: *** cfs_fail_loc=159, val=0*** [ 3977.163161] Lustre: lustre-MDT0000: Client 447c4aaa-5ccf-4e5b-8605-68f901ce233e (at 192.168.206.26@tcp) reconnecting [ 3979.603525] Lustre: *** cfs_fail_loc=159, val=0*** [ 3996.083155] Lustre: lustre-MDT0001: Client 447c4aaa-5ccf-4e5b-8605-68f901ce233e (at 192.168.206.26@tcp) reconnecting [ 3997.579611] Lustre: *** cfs_fail_loc=159, val=0*** [ 3997.585193] Lustre: Skipped 3 previous similar messages [ 4014.063192] Lustre: lustre-MDT0001: Client 447c4aaa-5ccf-4e5b-8605-68f901ce233e (at 192.168.206.26@tcp) reconnecting [ 4019.094150] Lustre: 139194:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4021.992130] Lustre: *** cfs_fail_loc=159, val=0*** [ 4021.997899] Lustre: Skipped 4 previous similar messages [ 4037.604312] Lustre: lustre-MDT0000: Client 867ad808-d8be-472b-aa17-a7d912ab6aba (at 192.168.206.26@tcp) reconnecting [ 4039.316956] Lustre: *** cfs_fail_loc=159, val=0*** [ 4039.323861] Lustre: Skipped 14 previous similar messages [ 4055.548847] Lustre: lustre-MDT0000: Client 867ad808-d8be-472b-aa17-a7d912ab6aba (at 192.168.206.26@tcp) reconnecting [ 4057.400368] Lustre: 139486:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4063.724908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4063.728328] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4063.737556] LustreError: Skipped 2 previous similar messages [ 4063.775508] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4064.737995] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4064.739723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4069.231628] Lustre: server umount lustre-OST0000 complete [ 4074.980360] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.997215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.997783] Lustre: Skipped 1 previous similar message [ 4075.001442] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4075.005389] Lustre: Skipped 2 previous similar messages [ 4079.407538] Lustre: server umount lustre-MDT0000 complete [ 4080.100313] LustreError: 137746:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4080.124557] LustreError: 137746:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4083.042145] LustreError: 136135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583679 with bad export cookie 6360318904207860975 [ 4083.049752] LustreError: 136135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4083.053781] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4083.402467] Lustre: server umount lustre-MDT0001 complete [ 4094.444908] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 4097.213180] Key type lgssc unregistered [ 4097.534540] LNet: 140552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4097.544442] LNetError: 140552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4097.563466] LNet: Removed LNI 192.168.206.126@tcp [ 4098.367099] Key type .llcrypt unregistered [ 4098.368733] Key type ._llcrypt unregistered [ 4123.355145] Key type ._llcrypt registered [ 4123.362157] Key type .llcrypt registered [ 4123.540762] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4124.688779] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4124.757757] alg: No test for adler32 (adler32-zlib) [ 4126.564758] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 4126.899417] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 4128.600189] Key type lgssc registered [ 4129.699880] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4138.817701] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4138.845446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4140.529333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4145.236916] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4150.760975] LustreError: 141966:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4150.778848] LustreError: 141966:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4153.984494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4154.498443] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4160.597858] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4168.487858] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4173.956030] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4181.176056] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4181.472316] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4186.622623] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4188.314116] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4196.149763] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4199.934917] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4205.527987] Lustre: 144550:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4209.482451] Lustre: 144599:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4211.781044] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4213.119141] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4217.673146] Lustre: 144746:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4219.025660] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4220.137895] Lustre: 144854:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4227.560414] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4227.564361] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4227.590157] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4230.631635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4230.637541] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4230.645975] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4232.607906] Lustre: server umount lustre-OST0000 complete [ 4240.866760] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4240.879619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4240.889791] Lustre: Skipped 2 previous similar messages [ 4243.065231] Lustre: server umount lustre-MDT0000 complete [ 4245.993402] LustreError: 141965:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4246.010858] LustreError: 141965:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4247.520679] LustreError: 143808:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583844 with bad export cookie 16928353064630427465 [ 4247.533727] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4247.935209] Lustre: server umount lustre-MDT0001 complete [ 4259.450475] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 4263.290856] Key type lgssc unregistered [ 4263.707539] LNet: 145917:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4263.716440] LNetError: 145917:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4263.742286] LNet: Removed LNI 192.168.206.126@tcp [ 4264.868686] Key type .llcrypt unregistered [ 4264.871757] Key type ._llcrypt unregistered [ 4292.321454] Key type ._llcrypt registered [ 4292.323867] Key type .llcrypt registered [ 4292.465511] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4293.817693] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4293.879163] alg: No test for adler32 (adler32-zlib) [ 4295.009907] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 4295.292436] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 4296.970650] Key type lgssc registered [ 4298.270988] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4305.918976] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4305.943240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4307.468058] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4312.082185] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4317.673253] LustreError: 147334:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4317.697211] LustreError: 147334:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4320.546729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4320.890715] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4325.358396] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4332.388709] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4337.922797] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4343.563776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4343.854335] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4345.907571] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4349.497624] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4353.085984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4356.861836] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4362.953808] Lustre: *** cfs_fail_loc=159, val=0*** [ 4378.593042] Lustre: lustre-MDT0000: Client 27ff1b4f-4919-4630-9ec3-1920adb8f4fa (at 192.168.206.26@tcp) reconnecting [ 4383.725248] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4383.737173] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4383.741181] Lustre: Skipped 1 previous similar message [ 4383.755810] Lustre: Skipped 1 previous similar message [ 4385.836637] Lustre: server umount lustre-OST0000 complete [ 4392.417085] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4392.426557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4392.443036] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4393.954026] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4393.956222] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4396.514499] Lustre: server umount lustre-MDT0000 complete [ 4399.083864] LustreError: 147335:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4399.098645] LustreError: 147335:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4400.257554] LustreError: 147315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787583997 with bad export cookie 17408572691512937819 [ 4400.264829] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4400.269596] LustreError: 147315:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4400.719625] Lustre: server umount lustre-MDT0001 complete [ 4411.501479] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 4414.388407] Key type lgssc unregistered [ 4414.723810] LNet: 150998:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4414.737955] LNetError: 150998:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4414.767802] LNet: Removed LNI 192.168.206.126@tcp [ 4415.550310] Key type .llcrypt unregistered [ 4415.552425] Key type ._llcrypt unregistered [ 4438.842339] Key type ._llcrypt registered [ 4438.844732] Key type .llcrypt registered [ 4438.910895] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4439.833439] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4439.969803] alg: No test for adler32 (adler32-zlib) [ 4441.035397] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 4441.246735] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 4442.896247] Key type lgssc registered [ 4443.793636] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4450.984744] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4451.002723] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4452.537429] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4456.638376] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4463.080380] LustreError: 152415:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4463.103682] LustreError: 152415:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4463.952782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4464.334825] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4468.164559] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4474.717031] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4479.251626] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4485.322892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4485.768614] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4489.915929] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4491.277246] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4498.115369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4498.273932] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4518.006743] Lustre: lustre-MDT0001: evicting 700ea82d-c2c4-4db0-9153-32b7a16c5269 (at 192.168.206.26@tcp) by administrative request [ 4518.020163] Lustre: Skipped 1 previous similar message [ 4547.560920] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4547.565282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4547.570761] Lustre: Skipped 1 previous similar message [ 4547.577867] Lustre: Skipped 1 previous similar message [ 4552.676783] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4552.679619] Lustre: Skipped 1 previous similar message [ 4557.804552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4560.864181] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4561.001600] Lustre: server umount lustre-OST0000 complete [ 4567.010317] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4567.021581] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4567.032724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4567.038463] Lustre: Skipped 1 previous similar message [ 4568.035581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4570.722326] Lustre: server umount lustre-MDT0000 complete [ 4573.154913] LustreError: 154822:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4573.170627] LustreError: 154822:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4573.770266] LustreError: 152396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584170 with bad export cookie 7432668810917291627 [ 4573.776842] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4573.782478] LustreError: 152396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4574.028107] Lustre: server umount lustre-MDT0001 complete [ 4582.990235] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 4585.281907] Key type lgssc unregistered [ 4585.532306] LNet: 156489:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4585.542875] LNetError: 156489:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4585.567669] LNet: Removed LNI 192.168.206.126@tcp [ 4586.271878] Key type .llcrypt unregistered [ 4586.273886] Key type ._llcrypt unregistered [ 4658.185791] Key type ._llcrypt registered [ 4658.191166] Key type .llcrypt registered [ 4658.332410] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4660.104280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4660.149191] alg: No test for adler32 (adler32-zlib) [ 4661.499367] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 4662.174799] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 4663.944598] Key type lgssc registered [ 4664.989256] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4674.187285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4674.238132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4676.101627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4680.928648] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4685.283720] LustreError: 158976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4685.297755] LustreError: 158976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4689.145328] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4689.548905] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4692.317573] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4699.579496] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4709.191894] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4720.561539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4721.548692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4732.354990] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4734.837331] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4740.121205] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4746.106566] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4755.435951] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4755.443839] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4755.465935] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4756.458483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4756.459759] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4761.231329] Lustre: server umount lustre-OST0000 complete [ 4766.178113] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4766.185817] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4766.209260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4766.214797] Lustre: Skipped 1 previous similar message [ 4771.814046] LustreError: 158976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4771.833115] Lustre: server umount lustre-MDT0000 complete [ 4771.834023] LustreError: 158976:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4775.722630] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584372 with bad export cookie 9547735528056920677 [ 4775.739022] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4775.739406] LustreError: 158957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4775.953827] Lustre: server umount lustre-MDT0001 complete [ 4788.768480] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 4791.928589] Key type lgssc unregistered [ 4792.239548] LNet: 162543:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4792.249392] LNetError: 162543:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4792.282464] LNet: Removed LNI 192.168.206.126@tcp [ 4793.295426] Key type .llcrypt unregistered [ 4793.297741] Key type ._llcrypt unregistered [ 4818.418526] Key type ._llcrypt registered [ 4818.422278] Key type .llcrypt registered [ 4818.567487] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4819.659689] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4819.794721] alg: No test for adler32 (adler32-zlib) [ 4820.915723] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 4821.149748] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 4822.809125] Key type lgssc registered [ 4823.979973] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4839.249753] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:13:53 (1787584433) [ 4855.890452] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 4868.773549] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 4905.604080] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4905.634326] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4906.940374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4906.960039] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4907.041708] Lustre: lustre-MDT0000: new disk, initializing [ 4907.148404] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4907.188316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4912.137281] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4923.948805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4924.108690] Lustre: 166810:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4924.146484] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4924.149798] Lustre: Skipped 1 previous similar message [ 4924.198569] Lustre: lustre-MDT0001: new disk, initializing [ 4924.275194] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4924.336804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4924.342297] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4928.695621] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4938.185270] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4943.989839] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4951.223750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4951.579418] Lustre: lustre-OST0000: new disk, initializing [ 4951.582502] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4951.588362] Lustre: 168779:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4951.678351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4957.820507] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 4960.844087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4960.865291] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4960.977427] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4968.396331] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4981.731507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4981.739658] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4981.761328] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4981.770609] Lustre: Skipped 1 previous similar message [ 4984.342675] Lustre: server umount lustre-OST0000 complete [ 4990.945894] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4990.951173] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4990.961914] Lustre: Skipped 1 previous similar message [ 4990.972187] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4991.972208] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4991.989981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4991.994759] Lustre: Skipped 1 previous similar message [ 4994.485224] Lustre: server umount lustre-MDT0000 complete [ 4998.384318] LustreError: 166802:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787584595 with bad export cookie 3871076338684833247 [ 4998.390620] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4998.392766] LustreError: 166802:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4998.688665] Lustre: server umount lustre-MDT0001 complete [ 5010.377278] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 5013.342830] Key type lgssc unregistered [ 5013.697881] LNet: 170685:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5013.702779] LNetError: 170685:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5014.761680] LNet: Removed LNI 192.168.206.126@tcp [ 5015.802684] Key type .llcrypt unregistered [ 5015.806979] Key type ._llcrypt unregistered [ 5039.286090] Key type ._llcrypt registered [ 5039.288899] Key type .llcrypt registered [ 5039.377992] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 5040.379942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5040.484072] alg: No test for adler32 (adler32-zlib) [ 5041.613192] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 5041.866726] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 5043.529399] Key type lgssc registered [ 5044.617092] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5051.061649] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5051.078152] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5052.413967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5055.790941] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 5062.625412] LustreError: 172101:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5062.645530] LustreError: 172101:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5063.616474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5064.039837] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5068.511516] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 5076.251658] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5082.286877] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5088.806449] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5089.056617] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5095.443680] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 5097.295997] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5103.474937] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5112.378712] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5112.389811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5112.415740] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5112.422035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.435098] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5113.373923] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5113.386936] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5113.397132] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5113.402924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.425369] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5114.354207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.253468] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5115.258211] Lustre: Skipped 1 previous similar message [ 5115.269329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5115.289405] Lustre: Skipped 1 previous similar message [ 5115.296262] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5115.308119] Lustre: Skipped 1 previous similar message [ 5115.315908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.330593] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5115.343235] Lustre: Skipped 1 previous similar message [ 5116.267728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.019594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.797964] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5117.808771] Lustre: Skipped 2 previous similar messages [ 5117.824559] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5117.832957] Lustre: Skipped 2 previous similar messages [ 5117.836671] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5117.840083] Lustre: Skipped 2 previous similar messages [ 5117.842984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.849950] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5117.857976] Lustre: Skipped 2 previous similar messages [ 5118.665865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.596131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.406695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.299667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.049518] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5122.054831] Lustre: Skipped 4 previous similar messages [ 5122.068377] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5122.091390] Lustre: Skipped 4 previous similar messages [ 5122.097216] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5122.105674] Lustre: Skipped 4 previous similar messages [ 5122.110531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.122359] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5122.130831] Lustre: Skipped 4 previous similar messages [ 5122.911578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.639831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.512841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.349403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.206751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.026671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.931278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.829601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.759447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.501287] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5130.504765] Lustre: Skipped 9 previous similar messages [ 5130.516787] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5130.523715] Lustre: Skipped 9 previous similar messages [ 5130.529974] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5130.533427] Lustre: Skipped 9 previous similar messages [ 5130.538883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.549585] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5130.555721] Lustre: Skipped 9 previous similar messages [ 5131.329447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.186706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.986333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.883216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.721497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.615428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.674184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.815646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.789511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.665493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.552339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.460460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.590391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.705718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.887339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.723856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.865846] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5146.870378] Lustre: Skipped 16 previous similar messages [ 5146.885302] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5146.896384] Lustre: Skipped 16 previous similar messages [ 5146.899852] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5146.903414] Lustre: Skipped 16 previous similar messages [ 5146.907665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.917899] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5146.920445] Lustre: Skipped 16 previous similar messages [ 5148.059473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.249503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.087992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.291710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.298760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.228870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.209124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.359464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.364709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.404936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.189585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.441875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.146263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.992740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.782502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.716426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.921245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.832547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.570663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.420065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.218464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.142279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.008564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.882987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.659243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.565922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.470976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.401346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.197052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.211331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.188022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.058909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.941748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.920014] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5178.928528] Lustre: Skipped 33 previous similar messages [ 5178.945857] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5178.964697] Lustre: Skipped 33 previous similar messages [ 5178.974472] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5178.980326] Lustre: Skipped 33 previous similar messages [ 5178.988719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.003261] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5179.010552] Lustre: Skipped 33 previous similar messages [ 5179.907740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.819864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.640159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.543569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.256544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.177605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.988739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.854912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.631158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.720308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.655976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.593740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.650982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.491715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.345994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.126448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.028623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.880423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.719968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.564427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.338959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.193993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.152454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.085941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.102144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.160543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.065983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.939697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.724199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.615474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.590683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.502393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.365201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.062435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.849658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.590379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.555918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.445257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.255539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.177341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.182509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.098834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.940918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.880521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.696150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.562192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.516857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.480723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.440638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.335155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.234995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.035828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.995441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.817466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.817303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.759501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.654976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.581925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.482509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.447302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.450057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.362986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.240205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.977334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.796409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.481566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.326977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.177162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.993959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.695574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.349625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.204669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.970203] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5242.977460] Lustre: Skipped 72 previous similar messages [ 5242.993916] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5243.006222] Lustre: Skipped 72 previous similar messages [ 5243.009932] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5243.014043] Lustre: Skipped 72 previous similar messages [ 5243.018037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.024728] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5243.030416] Lustre: Skipped 72 previous similar messages [ 5243.696244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.413989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.072318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.784795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.560442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.298149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.082899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.733288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.431134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.041939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.857953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.516319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.203851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.804311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.685371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.396804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.102948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.863843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.546290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.424903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.346667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.192356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.984204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.776637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.786696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.621679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.693397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.654712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.986608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.828775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.707758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.603468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.455713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.437740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.414679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.332064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.986199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.866678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.778780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.501814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.666844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.808532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.697859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.712564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.471257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.539702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.461457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.497886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.540192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.612031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.576115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.441351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.251181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.211909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.053519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.816442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.729507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.501249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.499977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.423580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.318459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.205156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.075395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.889610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.767090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.731920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.631520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.570444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.533403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.533053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.533406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.510609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.465877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.279842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.265709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.326945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.330255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.549481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.628922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.513617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.358076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.227200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.013865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.828916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.816607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.748435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.708126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.865221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.917499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.985049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.956984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.881979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.216812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.257651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.366321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.166958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.925643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.753527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.690415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.491453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.262183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.077349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.972958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.843794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.810477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.591753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.924576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.121141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.226697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.316635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.217289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.123308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.084487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.149888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.192177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.046847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.813131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.747490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.496883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.526179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.203291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.143351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.052925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.758561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.534312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.310284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.061677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.800317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.610593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.555428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.287107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.219481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.119147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.876350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.905277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.031297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.938753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.742055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.530970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.549478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.370135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.210202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.059524] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5371.067213] Lustre: Skipped 142 previous similar messages [ 5371.088763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5371.098181] Lustre: Skipped 142 previous similar messages [ 5371.101727] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5371.107249] Lustre: Skipped 142 previous similar messages [ 5371.112354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.123344] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5371.139294] Lustre: Skipped 142 previous similar messages [ 5372.182293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.038846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.770962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.476677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.469291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.264442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.255968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.182962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.099971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.767476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.503252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.681920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.372870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.234213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.997953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.698127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.623529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.427841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.308418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.222045] LustreError: 172120:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5388.228393] LustreError: 172120:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5388.247168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.245452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.150727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.895814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.887368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.933378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.807867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.576356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.427067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.399789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.361150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.477407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.906835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.494644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.663526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.795477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.837593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.980171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.802617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.723804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.625991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.891551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.940948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.778332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.552568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.509158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.387573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.335924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.297265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.333352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.194759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.177802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.091963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.943813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.777627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.608995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.521525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.386553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.171596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.964375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.774817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.556554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.508861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.458959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.446141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.292107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.111312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.899600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.667555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.535512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.397896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.550900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.868663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.846620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.908806] LustreError: 172120:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5439.922326] LustreError: 172120:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5439.924884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.751645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.783837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.659355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.500995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.403440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.518213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.540263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.775650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.794275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.897257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.098512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.089658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.117266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.977200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.004809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.911919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.887692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.779729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.754721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.619643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.640900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.447309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.417792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.466194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.296803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.488237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.561962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.543228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.753149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.821155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.916227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.821774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.875675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.813221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.699657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.614583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.782636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.672218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.803492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.833208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.956439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.084465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.995305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.118867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.104185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.059549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.979423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.041528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.867787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.852860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.834741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.745843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.721841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.774534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.652992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.369786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.265985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.102792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.888188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.843325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.682738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.790823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.653338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.529168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.377377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.302659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.399628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.292039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.140206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.028389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.014402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.996506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.933722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.773887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.670887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.690431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.693668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.742526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.754818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.743640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.709790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.776463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.943987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.036835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.994735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.137836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.059323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.068185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.032878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.029721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.037815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.099912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.000423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.985146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.018389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.031600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.881866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.931879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.826579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.847689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.698222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.696314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.711463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.619543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.545127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.387298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.326336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.238950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.110084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.974214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.165085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.183556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.194926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.381879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.290355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.385621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.368360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.632480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.644253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.734735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.623275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.835677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.983906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.890508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.028176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.016923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.950544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.881188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.043111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.238923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.317270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.421423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.389559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.531435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.554811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.548978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.611377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.409211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.603609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.510394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.757591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.995594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.108422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.004703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.200048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.263368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.098702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.040170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.859140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.631426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.520870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.449922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.453242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.279784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.042962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.766767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.677423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.618303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.501462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.302738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.169904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.974060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.981856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.763958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.601427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.592721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.616910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.754527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.752268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.792503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.711268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.758474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.759882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.721735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.526433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.368662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.235465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.921270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.750864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.722772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.528833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.235339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.015707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.804628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.625836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.516526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.257787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.265459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.304834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.196787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.018291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.832361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.840354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.749180] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5627.751670] Lustre: Skipped 267 previous similar messages [ 5627.762672] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5627.771390] Lustre: Skipped 267 previous similar messages [ 5627.779802] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5627.784127] Lustre: Skipped 267 previous similar messages [ 5627.793268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.804751] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5627.813177] Lustre: Skipped 267 previous similar messages [ 5628.673138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.554343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.438972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.373877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.390135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.361668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.345498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.304312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.068406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.989854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.876520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.799267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.631771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.522227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.385904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.245444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.285678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.260543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.133486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.027318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.868181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.766758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.511379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.418196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.318561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.223681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.970895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.856166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.841810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.793837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.616697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.532124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.384338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.310663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.099598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.839043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.516052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.435345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.128287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.864381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.686621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.457397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.251314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.192356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.886547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.711092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.412116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.147509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.905154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.767696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.600560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.431973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.151823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.995184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.755107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.522156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.409208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.357066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.303790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.331836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.186532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.002334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.825339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.710072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.556793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.396407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.206479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.895419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.768481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.603873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.451356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.244398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.083365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.170934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.038109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.148877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.046829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.027568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.920146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.838042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.622153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.457765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.441423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.387789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.282403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.149770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.021460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.666603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.542088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.378788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.179536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.042867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.778488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.594586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.410281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.226862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.124401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.080515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.922389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.808340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.480138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.259559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.068034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.195924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.970316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.847881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.601058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.406327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.293592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.349879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.335848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.353675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.598772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.675281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.627863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.488933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.528313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.515780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.543193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.330893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.347198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.411240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.384261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.491375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.470740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.630316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.557439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.512864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.303314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.287405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.019082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.911595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.642283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.452280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.245448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.143521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.901398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.707925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.642947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.439824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.241173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.048533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.939618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.791505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.439594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.171776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.238745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.175853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.331825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.137805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.941557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.899275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.916318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.910335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.835520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.718361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.633151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.513173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.555098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.450557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.350662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.287859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.173614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.969677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.991680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.868035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.659576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.474254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.334804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.053261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.764885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.440560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.274431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.042922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.805762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.692137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.511760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.386898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.227760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.240544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.090343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.158804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.119248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.174717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.134487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.317246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.538216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.884977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.243626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.729519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.050488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.962985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.092240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.266734] LustreError: 172120:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5801.278352] LustreError: 172120:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5801.302042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.490468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.424286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.536890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.861558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.066134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.744772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.872439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.192727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.115207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.336609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.123639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.969890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.866630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.557737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.268656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.946106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.828119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.711874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.570363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.557993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.500330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.321493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.144182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.875968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.907309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.907801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.666138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.506845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.390641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.107340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.750557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.428480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.116279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.758297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.471410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.336723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.123041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.854301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.608838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.394645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.179346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.939666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.792701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.648555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.470899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.286159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.235555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.367371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.323597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.250369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.000528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.722950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.392710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.191479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.881873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.578952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.342168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.127094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.970431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.850582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.700337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.618249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.411484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.181616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.922277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.730412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.640084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.446145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.394787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.255614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.080843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.751591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.492508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.378024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.215903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.126790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.990096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.844801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.707471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.732795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.519795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.365545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.362121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.225780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.997290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.883576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.712126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.593508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.338504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.177608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.895446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.584395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.316316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.153748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.814043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.563856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.300235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.148776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.020286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.747643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.475740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.129981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.816925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.469202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.159264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.840126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.591886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.369667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.103717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.835153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.574450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.383748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.193587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.082074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.934181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.853509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.772540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.601210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.426215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.361485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.240506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.170101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.029487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.789184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.713223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.561829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.393493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.019360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.849173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.802290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.457265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.214944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.994554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.692689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.888266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.070444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.901610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.739953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.442190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.380519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.240671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.094803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.018351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.945969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.096056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.500497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.851368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.023240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.467704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.684740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.931571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.229425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.510083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.808199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.767556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.530270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.218746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.849704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.000321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.468041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.619845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.895064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.389292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.462520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.685280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.986121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.844456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.549398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.313065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.232734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.259468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.991497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.816062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.369831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.147386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.046891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.022551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.189138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.715905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.469851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.574478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.738463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.517596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.462667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.446073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.592850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.866521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.307457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.773296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.519969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.726534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.575244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.399553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.115427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.084468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.837967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.851375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.635599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.526892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.597373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.556470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.389187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.186923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.476732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.249813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.059147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.893528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.908033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.674604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.371532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.190427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.292546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.267035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.345565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.403250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.605459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.712406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.708137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.126082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.278126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.130876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.308863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.381278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.587473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.866484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.705268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.651896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.522092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.369299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.153198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.010579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.202231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.227321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.598030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.690874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.156839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.454973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.711792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.611309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.939605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.297405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.567288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.845584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.008610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.024026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.417675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.768952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.831215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.883919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.703145] LustreError: 173924:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6435.715650] LustreError: 172120:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7099.898306] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7099.902833] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7099.912828] LustreError: Skipped 1 previous similar message [ 7099.926157] Lustre: Skipped 443 previous similar messages [ 7099.932038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7101.416163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7101.421238] Lustre: Skipped 2 previous similar messages [ 7103.389831] Lustre: server umount lustre-OST0000 complete [ 7106.535923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7106.542860] Lustre: Skipped 1 previous similar message [ 7111.650338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7111.660770] Lustre: Skipped 1 previous similar message [ 7113.032775] Lustre: server umount lustre-MDT0000 complete [ 7116.558904] LustreError: 172081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586713 with bad export cookie 13660570619782044812 [ 7116.565273] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7116.569687] LustreError: 172081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7116.977475] Lustre: server umount lustre-MDT0001 complete [ 7129.957306] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 7133.708390] Key type lgssc unregistered [ 7134.144933] LNet: 227360:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7134.152232] LNetError: 227360:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7134.166853] LNet: Removed LNI 192.168.206.126@tcp [ 7135.377433] Key type .llcrypt unregistered [ 7135.379666] Key type ._llcrypt unregistered [ 7161.592653] Key type ._llcrypt registered [ 7161.595400] Key type .llcrypt registered [ 7161.690450] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7162.507268] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7162.527797] alg: No test for adler32 (adler32-zlib) [ 7163.528702] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 7163.740772] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 7165.432196] Key type lgssc registered [ 7166.496946] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7174.490604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7174.513215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7175.869181] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7179.893577] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7185.380674] LustreError: 228776:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7185.398617] LustreError: 228776:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7187.360947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7187.707766] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7191.793243] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7199.767947] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7205.441734] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7211.954881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7212.233752] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7217.977455] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7220.462313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7224.446565] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7230.013384] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7231.745174] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7231.764617] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7231.772315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.783320] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7231.785790] LustreError: 228796:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7234.025073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7234.040548] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7234.053967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7236.067564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7236.073604] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7236.091028] Lustre: Skipped 1 previous similar message [ 7237.573829] Lustre: lustre-OST0000: Not available for connect from 192.168.206.26@tcp (stopping) [ 7239.787493] Lustre: server umount lustre-OST0000 complete [ 7251.856150] LustreError: 228796:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7273.205588] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7273.576942] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7276.665327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7276.682069] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7279.164210] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7286.460650] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7286.954631] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7286.967477] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7287.866417] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7293.355742] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa1000726a800.ost_server_uuid 50 [ 7294.904521] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa1000726a800.ost_server_uuid in FULL state after 0 sec [ 7300.071371] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7300.089398] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7300.124536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7302.635314] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7305.323452] Lustre: server umount lustre-OST0000 complete [ 7310.829656] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7310.839810] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7310.851720] Lustre: Skipped 1 previous similar message [ 7310.861736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7310.872823] Lustre: Skipped 2 previous similar messages [ 7315.752263] Lustre: server umount lustre-MDT0000 complete [ 7317.987073] LustreError: 229829:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7318.008804] LustreError: 229829:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7319.766379] LustreError: 228756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787586916 with bad export cookie 16794136457663704393 [ 7319.777341] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7319.787140] LustreError: 228756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7320.141792] Lustre: server umount lustre-MDT0001 complete [ 7333.712390] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 7336.425756] Key type lgssc unregistered [ 7336.769292] LNet: 233985:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7336.776833] LNetError: 233985:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7336.818372] LNet: Removed LNI 192.168.206.126@tcp [ 7337.863154] Key type .llcrypt unregistered [ 7337.864590] Key type ._llcrypt unregistered [ 7366.981169] Key type ._llcrypt registered [ 7366.984326] Key type .llcrypt registered [ 7367.129424] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7368.456857] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7368.466841] alg: No test for adler32 (adler32-zlib) [ 7369.532684] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 7369.791336] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 7371.472188] Key type lgssc registered [ 7372.537621] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7381.731182] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7381.764864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7383.215936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7387.941266] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7393.763390] LustreError: 235715:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7393.784037] LustreError: 235715:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7397.334047] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7397.691993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7402.053752] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7405.545167] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7412.475645] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7412.884368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7419.718829] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7429.333588] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7429.373063] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7429.597320] Lustre: lustre-OST0001: new disk, initializing [ 7429.601846] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7429.613319] Lustre: 238028:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7429.726483] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7435.947084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7435.972256] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7436.114221] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7436.604104] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7448.453968] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7455.638990] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7469.247069] Lustre: server umount lustre-MDT0000 complete [ 7469.539975] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7469.544610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7469.555917] LustreError: 235716:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7472.100700] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7472.101068] LustreError: 235715:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7472.109687] Lustre: Skipped 1 previous similar message [ 7472.849206] LustreError: 235698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587069 with bad export cookie 6734804636359274006 [ 7472.860457] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7472.863254] LustreError: 235698:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7475.309872] Lustre: server umount lustre-MDT0001 complete [ 7490.094520] Lustre: server umount lustre-OST0000 complete [ 7503.938491] Lustre: server umount lustre-OST0001 complete [ 7523.094524] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7532.401411] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7532.792924] LustreError: 241133:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7532.812176] LustreError: 241133:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7532.887955] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7537.788959] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7538.147332] LustreError: 241134:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7546.735748] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7547.202430] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7551.619100] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7554.592054] Lustre: 242275:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7560.558302] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7565.942285] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7571.118362] LustreError: 242628:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7571.135063] LustreError: 242628:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7572.704103] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7572.939418] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7572.949652] Lustre: Skipped 1 previous similar message [ 7578.037874] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7578.335218] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7585.656867] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7589.657147] Lustre: 244146:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7603.687765] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7603.698120] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7603.709374] Lustre: Skipped 3 previous similar messages [ 7603.711326] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7603.732674] Lustre: Skipped 3 previous similar messages [ 7606.916081] Lustre: server umount mylustre-MDT0000 complete [ 7608.802028] LustreError: 241134:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7608.834496] LustreError: 241134:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7611.057493] LustreError: 242652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587207 with bad export cookie 6734804636359276813 [ 7611.063834] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7611.069032] LustreError: 242652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7611.348893] Lustre: server umount mylustre-MDT0001 complete [ 7624.637735] Lustre: server umount mylustre-OST0000 complete [ 7638.700813] Lustre: server umount mylustre-OST0001 complete [ 7660.240638] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7673.702704] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7674.302777] LustreError: 246468:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7674.467814] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7679.736306] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7690.530730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7695.553737] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7698.285961] Lustre: 247610:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7705.495391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7711.846735] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7719.406545] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7720.754443] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7720.758971] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7725.134988] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7732.413201] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7735.674238] Lustre: 249486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7745.824760] Lustre: server umount tfs-MDT0000 complete [ 7746.549166] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7746.560766] LustreError: 246463:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7746.585074] Lustre: Skipped 3 previous similar messages [ 7746.604629] LustreError: 246463:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7749.661125] LustreError: 246445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587346 with bad export cookie 6734804636359278479 [ 7749.663235] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7749.691033] LustreError: 246445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7750.212167] Lustre: server umount tfs-MDT0001 complete [ 7763.378639] Lustre: server umount tfs-OST0000 complete [ 7777.329277] Lustre: server umount tfs-OST0001 complete [ 7795.903365] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7804.007748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7804.385537] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7804.394958] Lustre: Skipped 3 previous similar messages [ 7807.802347] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7815.593761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7820.770863] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7824.231704] Lustre: 252878:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7830.767706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7837.063620] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7844.991032] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7849.352689] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7849.356325] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 7850.664735] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 7859.274209] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7863.004805] Lustre: 254749:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7870.342940] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:04:24 (1787587464) [ 7887.841470] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7887.845387] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7887.851677] Lustre: Skipped 1 previous similar message [ 7887.854596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7890.416587] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7890.428765] Lustre: Skipped 3 previous similar messages [ 7892.202492] Lustre: server umount lustre-MDT0000 complete [ 7895.300446] LustreError: 251713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587492 with bad export cookie 6734804636359280145 [ 7895.302854] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7895.309120] LustreError: 251713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7895.523323] LustreError: 255012:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7895.528267] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7895.545829] LustreError: 255012:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 7895.557146] Lustre: Skipped 1 previous similar message [ 7900.650962] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7900.661407] Lustre: Skipped 1 previous similar message [ 7901.672696] Lustre: server umount lustre-MDT0001 complete [ 7911.386645] Lustre: server umount lustre-OST0000 complete [ 7929.312116] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7929.574397] Lustre: server umount lustre-OST0001 complete [ 7936.881897] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 7947.196888] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 7986.681137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7986.883639] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7986.912616] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7987.003929] Lustre: lustre-MDT0000: new disk, initializing [ 7987.133764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7987.144158] Lustre: Skipped 3 previous similar messages [ 7987.162757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7991.621635] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8004.236223] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8004.418112] Lustre: 257937:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8004.504973] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8004.510230] Lustre: Skipped 1 previous similar message [ 8004.628547] Lustre: lustre-MDT0001: new disk, initializing [ 8004.856307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8004.874616] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8009.571316] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8019.503406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8019.733365] Lustre: lustre-OST0000: new disk, initializing [ 8019.735827] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8019.740556] Lustre: 259519:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8021.440417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8021.456938] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8021.588994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8027.122953] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8043.794059] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8043.981572] Lustre: lustre-OST0001: new disk, initializing [ 8043.988162] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8043.996744] Lustre: 260391:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8045.867300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8045.889038] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8046.042986] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8051.598345] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8066.152386] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:07:40 (1787587660) [ 8083.013495] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:07:56 (1787587676) [ 8098.273597] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8098.273701] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8098.281416] LustreError: Skipped 1 previous similar message [ 8098.283825] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8098.293843] Lustre: Skipped 6 previous similar messages [ 8098.309650] Lustre: Skipped 1 previous similar message [ 8104.455663] Lustre: server umount lustre-OST0000 complete [ 8112.621804] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8114.728104] Lustre: server umount lustre-MDT0000 complete [ 8118.188222] LustreError: 257928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587715 with bad export cookie 6734804636359281699 [ 8118.197679] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8118.221234] LustreError: 257928:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8122.857400] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8122.865603] Lustre: Skipped 3 previous similar messages [ 8122.872121] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8122.877021] Lustre: Skipped 5 previous similar messages [ 8124.778563] Lustre: server umount lustre-MDT0001 complete [ 8135.472444] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 8209.471911] Lustre: server umount lustre-OST0001 complete [ 8217.270739] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 8233.477758] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 8271.988210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8272.410509] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8272.498698] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8272.639561] Lustre: lustre-MDT0000: new disk, initializing [ 8272.734620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8272.744591] Lustre: Skipped 3 previous similar messages [ 8272.764466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8278.009991] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8289.381363] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8289.548765] Lustre: 265250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8289.623536] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8289.626345] Lustre: Skipped 1 previous similar message [ 8289.699901] Lustre: lustre-MDT0001: new disk, initializing [ 8289.842368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8289.858608] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8294.460818] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8304.953962] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8312.346196] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8319.772854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8320.055230] Lustre: 267219:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8321.586841] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8321.712932] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8327.326503] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8338.083981] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8347.105719] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8347.113168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8347.128817] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8347.130607] Lustre: Skipped 1 previous similar message [ 8351.465038] Lustre: server umount lustre-OST0000 complete [ 8361.705477] Lustre: server umount lustre-MDT0000 complete [ 8362.980643] LustreError: 267227:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8362.991771] LustreError: 267227:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 12 previous similar messages [ 8365.142903] LustreError: 266045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787587962 with bad export cookie 6734804636359283169 [ 8365.157720] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8365.161241] LustreError: 266045:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8365.622338] Lustre: server umount lustre-MDT0001 complete [ 8377.570750] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 8381.159526] Key type lgssc unregistered [ 8381.514856] LNet: 268985:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8381.533500] LNetError: 268985:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8381.558590] LNet: Removed LNI 192.168.206.126@tcp [ 8382.666292] Key type .llcrypt unregistered [ 8382.669874] Key type ._llcrypt unregistered [ 8417.629868] Key type ._llcrypt registered [ 8417.633619] Key type .llcrypt registered [ 8417.778266] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 8419.056965] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8419.161061] alg: No test for adler32 (adler32-zlib) [ 8420.334787] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 8420.586053] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 8422.312166] Key type lgssc registered [ 8423.586843] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8433.087567] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8433.101525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8434.630681] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8439.282307] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8444.914658] LustreError: 270450:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8444.925334] LustreError: 270450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8448.888723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8449.248840] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8454.098596] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8462.034542] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8468.412455] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8477.386381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8477.772939] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8479.860746] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8483.853619] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8491.502435] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8492.515637] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8495.395235] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8495.412676] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8500.718608] LustreError: 272273:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8500.738598] LustreError: 272273:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8500.891438] Lustre: server umount lustre-OST0000 complete [ 8505.825562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8505.826463] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8505.842333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8505.842339] Lustre: Skipped 1 previous similar message [ 8505.889797] Lustre: Skipped 1 previous similar message [ 8510.945961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8510.953760] Lustre: Skipped 2 previous similar messages [ 8511.318744] Lustre: server umount lustre-MDT0000 complete [ 8515.355927] LustreError: 272294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787588112 with bad export cookie 4740011630447961345 [ 8515.376039] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8515.376493] LustreError: 272294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8515.773976] Lustre: server umount lustre-MDT0001 complete [ 8527.153291] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 8530.103664] Key type lgssc unregistered [ 8530.345630] LNet: 274040:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8530.354400] LNetError: 274040:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8530.382362] LNet: Removed LNI 192.168.206.126@tcp [ 8531.247360] Key type .llcrypt unregistered [ 8531.249086] Key type ._llcrypt unregistered [ 8547.410580] Key type ._llcrypt registered [ 8547.412352] Key type .llcrypt registered [ 8547.533851] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 8548.439653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8548.575564] alg: No test for adler32 (adler32-zlib) [ 8549.740324] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 8550.068560] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 8551.784200] Key type lgssc registered [ 8552.992231] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8562.121991] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8562.134335] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8577.696381] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8582.816421] LustreError: 275234:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.126@tcp: failed processing log, type 4: rc = -110 [ 8609.504157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8615.679893] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8622.624818] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8624.827809] Lustre: server umount lustre-OST0000 complete [ 8640.247720] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 8642.933475] Key type lgssc unregistered [ 8643.256162] LNet: 276868:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8643.264324] LNetError: 276868:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8643.279858] LNet: Removed LNI 192.168.206.126@tcp [ 8643.996789] Key type .llcrypt unregistered [ 8643.999476] Key type ._llcrypt unregistered [ 8681.997624] Key type ._llcrypt registered [ 8681.999352] Key type .llcrypt registered [ 8682.100591] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 8682.837710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8682.858531] alg: No test for adler32 (adler32-zlib) [ 8683.907382] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 8684.160838] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 8685.832209] Key type lgssc registered [ 8687.373132] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8702.042696] loop0: detected capacity change from 0 to 204800000 [ 8702.287914] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8702.301183] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8702.666753] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8703.735454] loop0: detected capacity change from 0 to 204800000 [ 8703.867710] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8703.880390] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8704.555882] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8705.842568] loop0: detected capacity change from 0 to 204800000 [ 8706.056777] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8706.071692] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8706.143369] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8707.124911] loop0: detected capacity change from 0 to 204800000 [ 8707.248947] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8707.371180] print_req_error: 1 callbacks suppressed [ 8707.371189] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8708.459245] loop0: detected capacity change from 0 to 204800000 [ 8708.484338] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8714.648568] loop0: detected capacity change from 0 to 204800000 [ 8714.707627] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8719.414247] loop0: detected capacity change from 0 to 204800000 [ 8719.437224] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8723.904354] loop0: detected capacity change from 0 to 204800000 [ 8723.967066] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8728.177179] loop0: detected capacity change from 0 to 204800000 [ 8728.278595] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8728.360750] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8728.471567] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8728.491633] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8728.509124] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8728.528468] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8728.545124] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8728.571124] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8728.583078] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8728.598586] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8728.618377] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8728.634986] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8728.646942] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8728.672135] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8728.689133] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8728.703397] Lustre: 280284:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8735.320974] Lustre: server umount MGS complete [ 8736.202977] loop0: detected capacity change from 0 to 204800000 [ 8736.267325] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8736.707351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8736.750818] Lustre: lustre-MDT0000: Aborting client recovery [ 8737.631651] loop1: detected capacity change from 0 to 204800000 [ 8737.817498] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8737.941897] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8737.959931] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8737.968550] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8737.978908] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8737.991580] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8738.009605] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8738.021741] Lustre: 280717:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8738.040557] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8738.230768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8738.287891] Lustre: lustre-MDT0001: Aborting client recovery [ 8739.224884] loop2: detected capacity change from 0 to 204800000 [ 8739.406176] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8739.486955] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8739.500496] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8739.508911] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8739.529265] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8739.540503] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8739.552735] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8739.559884] Lustre: 280796:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8739.684143] Lustre: Found index 0 for lustre-OST0000, updating log [ 8739.725383] Lustre: 280814:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8739.804508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8739.838648] Lustre: lustre-OST0000: Aborting recovery [ 8740.693405] loop3: detected capacity change from 0 to 204800000 [ 8740.856725] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8740.952427] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8740.960736] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8740.969359] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8740.977175] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8740.984748] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8740.999595] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8741.013752] Lustre: 280879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8741.067987] Lustre: Found index 1 for lustre-OST0001, updating log [ 8741.107957] Lustre: 280885:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8741.186714] Lustre: lustre-OST0001: Aborting recovery [ 8746.342760] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8746.390951] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8746.423823] LustreError: 280661:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8746.427962] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8746.484585] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8748.425671] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8748.431524] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8748.871444] LustreError: 280628:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 485us: rc = -2 [ 8749.400873] LustreError: 280892:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2654us: rc = -2 [ 8749.445134] LustreError: 280892:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 16 previous similar messages [ 8763.877639] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8763.880707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8763.881115] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8763.900221] Lustre: Skipped 2 previous similar messages [ 8768.566881] Lustre: server umount lustre-MDT0000 complete [ 8768.995542] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8769.014314] LustreError: 280627:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8769.468455] LustreError: 280612:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787588366 with bad export cookie 16542646140329430375 [ 8769.506575] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8774.115519] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8774.118665] LustreError: 280625:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8774.125277] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8774.125286] Lustre: Skipped 3 previous similar messages [ 8774.133073] Lustre: Skipped 2 previous similar messages [ 8774.161742] LustreError: 280625:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8776.105409] Lustre: server umount lustre-MDT0001 complete [ 8783.378741] Lustre: server umount lustre-OST0000 complete [ 8790.601257] Lustre: server umount lustre-OST0001 complete [ 8798.466974] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:19:52 (1787588392) [ 8826.987547] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 8836.582840] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 8873.501329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8873.712527] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8873.761697] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8873.849343] Lustre: lustre-MDT0000: new disk, initializing [ 8873.927924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8873.941687] Lustre: Skipped 1 previous similar message [ 8873.990539] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8879.687260] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8893.241426] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8893.342634] Lustre: 284952:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8893.427659] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8893.431586] Lustre: Skipped 1 previous similar message [ 8893.484823] Lustre: lustre-MDT0001: new disk, initializing [ 8893.637768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8893.675665] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8893.696795] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8898.993425] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8909.133485] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8916.004055] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8923.848293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8924.258780] Lustre: lustre-OST0000: new disk, initializing [ 8924.263497] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8924.269395] Lustre: 286919:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8924.356633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8930.367848] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8930.374755] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8930.525727] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8931.860229] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 8942.305912] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8947.996059] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8957.606091] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8961.463892] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8968.072088] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9017.831599] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9017.832893] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9017.833745] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9017.833751] Lustre: Skipped 1 previous similar message [ 9017.845023] LustreError: Skipped 1 previous similar message [ 9017.852504] Lustre: Skipped 1 previous similar message [ 9022.256439] Lustre: server umount lustre-OST0000 complete [ 9027.055570] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9027.068452] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9027.088822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9027.105438] Lustre: Skipped 1 previous similar message [ 9033.185281] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9033.187390] Lustre: Skipped 2 previous similar messages [ 9040.352158] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9040.550186] Lustre: server umount lustre-MDT0000 complete [ 9042.404979] LustreError: 284963:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9042.432278] LustreError: 284963:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9044.554544] LustreError: 285754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787588641 with bad export cookie 16542646140329433084 [ 9044.564875] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9044.573712] LustreError: 285754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9045.039944] Lustre: server umount lustre-MDT0001 complete [ 9054.377702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9059.451199] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9064.024444] Lustre: server umount MGS complete [ 9076.440291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9076.917779] LustreError: 290690:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9076.951852] LustreError: 290703:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9077.032355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9082.286926] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9082.338066] LustreError: 290691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9091.560323] LustreError: 290690:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9091.580351] LustreError: 290690:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9093.293559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9099.300870] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9108.199966] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9114.453746] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9121.312845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9121.590349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9121.596921] Lustre: Skipped 1 previous similar message [ 9127.602689] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9135.169932] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9180.129312] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9180.135917] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9180.152048] Lustre: Skipped 1 previous similar message [ 9180.156780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9180.165218] Lustre: Skipped 2 previous similar messages [ 9185.051789] Lustre: server umount lustre-OST0000 complete [ 9190.889627] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9190.909770] LustreError: Skipped 1 previous similar message [ 9190.918836] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9190.938579] Lustre: Skipped 1 previous similar message [ 9195.588364] Lustre: server umount lustre-MDT0000 complete [ 9197.544047] LustreError: 290691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9197.581112] LustreError: 290691:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9200.117720] LustreError: 290672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787588797 with bad export cookie 16542646140329434939 [ 9200.123225] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9200.129511] LustreError: 290672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9200.451791] Lustre: server umount lustre-MDT0001 complete [ 9213.348625] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 9216.725500] Key type lgssc unregistered [ 9217.004566] LNet: 294665:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9217.017373] LNetError: 294665:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9217.045498] LNet: Removed LNI 192.168.206.126@tcp [ 9218.203392] Key type .llcrypt unregistered [ 9218.207096] Key type ._llcrypt unregistered [ 9257.264180] Key type ._llcrypt registered [ 9257.274694] Key type .llcrypt registered [ 9257.378567] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 9272.826767] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 9273.885967] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9273.898725] alg: No test for adler32 (adler32-zlib) [ 9274.907041] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 9275.129423] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 9276.800972] Key type lgssc registered [ 9277.763871] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9314.801826] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9314.837024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9316.138606] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9316.196663] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9316.268992] Lustre: lustre-MDT0000: new disk, initializing [ 9316.327428] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9316.338729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9320.727271] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9331.757988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9331.834191] Lustre: 298622:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9331.886760] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9331.891290] Lustre: Skipped 1 previous similar message [ 9331.947817] Lustre: lustre-MDT0001: new disk, initializing [ 9332.019172] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9332.062580] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9332.078287] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9336.754132] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9348.543311] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9355.316474] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9362.859696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9363.141560] Lustre: lustre-OST0000: new disk, initializing [ 9363.147434] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9363.158766] Lustre: 300593:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9363.251568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9369.397624] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9372.761771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9372.775697] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9372.917535] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9380.002875] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9386.560365] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9400.415120] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9408.277718] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9414.974774] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9460.196638] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9460.204693] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9460.219724] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9460.226834] Lustre: Skipped 1 previous similar message [ 9464.000892] Lustre: server umount lustre-OST0000 complete [ 9470.439111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9470.442183] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9470.442353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9470.458649] Lustre: Skipped 2 previous similar messages [ 9470.470141] LustreError: Skipped 1 previous similar message [ 9475.554135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9475.561841] Lustre: Skipped 2 previous similar messages [ 9480.674085] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9480.686153] Lustre: Skipped 1 previous similar message [ 9482.209961] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9482.441513] Lustre: server umount lustre-MDT0000 complete [ 9486.529923] LustreError: 299427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787589083 with bad export cookie 13694755458893516091 [ 9486.533248] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9486.540020] LustreError: 299427:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9486.883227] Lustre: server umount lustre-MDT0001 complete [ 9496.254524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9501.341835] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9506.391985] Lustre: server umount MGS complete [ 9519.356745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9520.283748] LustreError: 304568:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9520.300511] LustreError: 304568:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9520.332590] LustreError: 304581:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9520.414552] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9525.024404] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9525.731785] LustreError: 304569:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9530.860418] LustreError: 304568:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9534.209529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9534.513429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9538.664942] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9547.272909] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9552.667741] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9560.540314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9560.927285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9567.826795] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9576.012643] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9607.137580] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9607.149741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9607.172365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9610.214276] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9610.217553] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9612.042651] Lustre: server umount lustre-OST0000 complete [ 9616.354352] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9616.367121] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9616.380557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9616.386387] Lustre: Skipped 2 previous similar messages [ 9622.292574] Lustre: server umount lustre-MDT0000 complete [ 9625.572444] LustreError: 304568:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9625.590349] LustreError: 304568:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9626.109930] LustreError: 307292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787589223 with bad export cookie 13694755458893518051 [ 9626.113651] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9626.123357] LustreError: 307292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9626.447566] Lustre: server umount lustre-MDT0001 complete [ 9638.544617] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [ 9641.830278] Key type lgssc unregistered [ 9642.094222] LNet: 308515:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9642.097733] LNetError: 308515:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9642.114725] LNet: Removed LNI 192.168.206.126@tcp [ 9643.010458] Key type .llcrypt unregistered [ 9643.012964] Key type ._llcrypt unregistered [ 9665.475762] Key type ._llcrypt registered [ 9665.477884] Key type .llcrypt registered [ 9665.569866] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [ 9680.769959] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 9681.705489] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9681.803476] alg: No test for adler32 (adler32-zlib) [ 9682.851575] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [ 9683.135569] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [ 9684.792146] Key type lgssc registered [ 9686.016102] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9712.124342] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [ 9727.963422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9728.005132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9729.257451] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9729.312501] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9729.384529] Lustre: lustre-MDT0000: new disk, initializing [ 9729.466877] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9729.485384] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9735.170760] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9746.932564] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9747.252568] Lustre: lustre-OST0000: new disk, initializing [ 9747.255710] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9747.260694] Lustre: Skipped 1 previous similar message [ 9747.268845] Lustre: 312569:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9747.394564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9755.518815] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [ 9756.262109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9756.268738] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9756.298429] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10667.489467] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10667.511943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10672.613599] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10673.618825] Lustre: server umount lustre-MDT0000 complete [10677.292773] LustreError: 311753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590274 with bad export cookie 17974212790816421889 [10677.297397] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10677.312318] LustreError: 311753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10677.549655] Lustre: server umount lustre-OST0000 complete [10704.731636] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [10715.179777] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [10754.445909] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10754.849238] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10754.892345] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10755.033689] Lustre: lustre-MDT0000: new disk, initializing [10755.129884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10755.159828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10759.130142] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10769.375146] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10769.559636] Lustre: 316911:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10769.595547] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10769.598457] Lustre: Skipped 1 previous similar message [10769.651963] Lustre: lustre-MDT0001: new disk, initializing [10769.749739] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10769.782812] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10769.796127] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10775.087515] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10785.629783] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10791.683607] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10799.183186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10799.534189] Lustre: lustre-OST0000: new disk, initializing [10799.541449] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10799.552703] Lustre: 318878:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10799.670157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10805.310637] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10805.321556] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10805.426161] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10805.723338] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10815.418679] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10821.130206] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10821.402542] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10825.984120] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10826.200383] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10831.332930] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10831.340485] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10831.364167] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10831.390061] Lustre: Skipped 1 previous similar message [10834.527403] Lustre: server umount lustre-OST0000 complete [10841.570346] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10841.572804] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10841.582365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10841.598926] Lustre: Skipped 2 previous similar messages [10846.689696] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10846.697201] Lustre: Skipped 2 previous similar messages [10853.344124] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10853.524514] Lustre: server umount lustre-MDT0000 complete [10856.880795] LustreError: 318888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590453 with bad export cookie 17974212790819362715 [10856.891160] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10856.896871] LustreError: 318888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10857.287332] Lustre: server umount lustre-MDT0001 complete [10860.712487] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10862.631082] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:54:16 (1787590456) [10869.589151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10870.051377] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10874.364251] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10875.363183] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10875.386727] LustreError: 321194:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10879.471185] LustreError: 321195:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10883.508728] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10883.996396] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10888.843693] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10896.880583] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10904.388987] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10912.326655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10912.669332] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10918.861696] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10927.538211] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10934.819366] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10935.014362] Lustre: lustre-OST0001: new disk, initializing [10935.017734] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10935.022669] Lustre: 324029:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10935.097825] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10941.590228] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [10942.649477] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10942.679421] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10942.744885] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10942.757842] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10943.786068] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10944.804557] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10944.818560] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10946.852623] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10946.871027] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10951.078230] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10951.105596] LustreError: 324427:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10952.367327] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10952.642841] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10958.793728] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa1001066f800.ost_server_uuid 50 [10959.406055] LustreError: 324442:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10959.416035] LustreError: 324442:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10960.256683] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa1001066f800.ost_server_uuid in FULL state after 0 sec [10965.343476] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa1001066f800.ost_server_uuid 50 [10966.826907] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa1001066f800.ost_server_uuid in FULL state after 0 sec [10974.186486] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10983.376580] Lustre: lustre-OST0001: Not available for connect from 192.168.206.26@tcp (stopping) [10983.385541] Lustre: Skipped 2 previous similar messages [10983.907767] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10983.911401] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10988.239387] Lustre: server umount lustre-OST0001 complete [10988.454893] LustreError: 324444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10994.155815] LustreError: 324444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10994.174181] LustreError: 324444:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10995.682105] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10995.692297] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10995.706347] Lustre: Skipped 1 previous similar message [10999.272315] LustreError: 324444:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10999.295082] LustreError: 324444:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11000.516086] Lustre: server umount lustre-OST0000 complete [11004.385796] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11004.405427] Lustre: Skipped 1 previous similar message [11004.413607] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11004.422139] Lustre: Skipped 7 previous similar messages [11010.555826] Lustre: server umount lustre-MDT0000 complete [11013.963910] LustreError: 321176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590610 with bad export cookie 17974212790819363597 [11013.965540] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11013.971167] LustreError: 321176:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11014.281429] Lustre: server umount lustre-MDT0001 complete [11028.971312] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [11033.102273] Key type lgssc unregistered [11033.473992] LNet: 326723:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11033.479323] LNetError: 326723:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11033.495973] LNet: Removed LNI 192.168.206.126@tcp [11034.431409] Key type .llcrypt unregistered [11034.433858] Key type ._llcrypt unregistered [11059.891333] Key type ._llcrypt registered [11059.903046] Key type .llcrypt registered [11060.083206] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [11061.554716] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11061.633654] alg: No test for adler32 (adler32-zlib) [11062.654483] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [11062.858471] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [11064.520260] Key type lgssc registered [11065.605911] Lustre: Echo OBD driver; http://www.lustre.org/ [11072.514855] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11072.531384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11073.935885] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11077.748263] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11083.241696] LustreError: 328146:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11083.268664] LustreError: 328146:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11084.381553] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11088.355405] LustreError: 328147:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11091.321336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11091.665868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11095.873817] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11105.470043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11105.853849] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11112.250657] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11117.125038] LustreError: 329825:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11119.145637] LustreError: 329824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11119.179371] LustreError: 329824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11119.200394] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11120.459192] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11124.198833] LustreError: 329824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11124.222336] LustreError: 329824:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11128.035802] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11128.382914] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11133.450389] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11134.854860] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11142.829602] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11149.570315] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11149.781472] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11256.315118] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11256.315656] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11256.346449] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11256.354647] Lustre: Skipped 1 previous similar message [11261.412938] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11261.417052] Lustre: Skipped 1 previous similar message [11262.134690] Lustre: server umount lustre-OST0001 complete [11266.531546] LustreError: 329826:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11266.572684] LustreError: 329826:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11270.625838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11270.636692] LustreError: Skipped 1 previous similar message [11270.644057] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11270.660268] Lustre: Skipped 1 previous similar message [11270.670429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11271.655564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11274.596294] Lustre: server umount lustre-OST0000 complete [11281.377994] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11281.388045] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11281.406225] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11281.412693] Lustre: Skipped 2 previous similar messages [11287.013328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11287.022021] Lustre: Skipped 1 previous similar message [11292.128162] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11292.360897] Lustre: server umount lustre-MDT0000 complete [11296.064649] LustreError: 331829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787590892 with bad export cookie 6709468771508531672 [11296.076648] LustreError: 331829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11296.077506] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11296.500151] Lustre: server umount lustre-MDT0001 complete [11308.730579] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [11311.467707] Key type lgssc unregistered [11311.807273] LNet: 333104:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11311.816707] LNetError: 333104:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11311.846580] LNet: Removed LNI 192.168.206.126@tcp [11312.789655] Key type .llcrypt unregistered [11312.792335] Key type ._llcrypt unregistered [11357.025953] Key type ._llcrypt registered [11357.029603] Key type .llcrypt registered [11357.136313] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [11358.291678] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11358.439034] alg: No test for adler32 (adler32-zlib) [11359.621634] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [11359.848065] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [11361.544764] Key type lgssc registered [11362.937643] Lustre: Echo OBD driver; http://www.lustre.org/ [11373.184493] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11373.220143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11374.792921] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11379.514530] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11384.299104] LustreError: 335397:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11384.313980] LustreError: 335397:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11388.007547] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11388.353328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11392.766277] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11395.889444] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11403.013189] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11403.534532] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11410.418681] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11413.808244] LustreError: 336886:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11418.717291] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11418.959070] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11420.036235] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11420.047252] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11424.418334] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11432.487315] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11441.468832] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11454.950858] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11454.953416] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11454.968165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11455.984785] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11455.987797] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11456.002735] Lustre: Skipped 2 previous similar messages [11456.022547] Lustre: Skipped 2 previous similar messages [11460.560812] Lustre: server umount lustre-MDT0000 complete [11461.099725] LustreError: 335392:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11461.118305] LustreError: 335392:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11464.147347] LustreError: 336133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591061 with bad export cookie 14224557661310149543 [11464.149161] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11464.165293] LustreError: 336133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11464.578364] Lustre: server umount lustre-MDT0001 complete [11478.806826] Lustre: server umount lustre-OST0000 complete [11492.656101] Lustre: server umount lustre-OST0001 complete [11507.974892] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [11517.776725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11518.091660] LustreError: 340661:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11518.156627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11521.928422] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11523.558451] LustreError: 340662:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11530.239655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11530.628430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11534.993919] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11537.881773] Lustre: 341800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11545.037154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11554.431806] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11555.644723] LustreError: 342152:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11555.686149] LustreError: 342152:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11563.039627] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11566.854502] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11567.081651] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11567.088655] Lustre: Skipped 1 previous similar message [11572.245911] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11574.077971] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11581.310669] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11585.917622] Lustre: 343675:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11592.164333] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11592.192599] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11592.208122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11592.216115] Lustre: Skipped 1 previous similar message [11596.686210] Lustre: server umount lustre-MDT0000 complete [11597.803896] LustreError: 341709:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11597.826320] LustreError: 341709:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [11600.835314] LustreError: 342984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591197 with bad export cookie 14224557661310151244 [11600.864339] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11600.864897] LustreError: 342984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11601.501389] Lustre: server umount lustre-MDT0001 complete [11616.006775] Lustre: server umount lustre-OST0000 complete [11618.914938] Lustre: 334658:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787591199/real 1787591199] req@ffffa071bc03b100 x1874424974395008/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787591215 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11618.941678] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11618.955111] Lustre: Skipped 3 previous similar messages [11620.110920] Lustre: server umount lustre-OST0001 complete [11623.726596] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11625.710687] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11627.745902] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:07:01 (1787591221) [11650.159553] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [11658.299762] loop0: detected capacity change from 0 to 1099511627776 [11658.744742] SGI XFS with ACLs, security attributes, quota, no debug enabled [11658.748839] *** VALIDATE xfs *** [11658.761474] XFS (loop0): Mounting V5 Filesystem [11658.793207] XFS (loop0): Ending clean mount [11660.271361] loop1: detected capacity change from 0 to 18691697672192 [11757.343726] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [11767.761829] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [11802.220101] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11802.518474] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11802.549743] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11802.623346] Lustre: lustre-MDT0000: new disk, initializing [11802.707743] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11802.726745] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11806.751750] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11816.816878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11816.934883] Lustre: 348807:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11816.991853] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11816.995565] Lustre: Skipped 1 previous similar message [11817.075483] Lustre: lustre-MDT0001: new disk, initializing [11817.198338] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11817.203829] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11821.287519] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11830.421327] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11835.532509] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11841.186943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11841.406821] Lustre: lustre-OST0000: new disk, initializing [11841.412301] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11841.426728] Lustre: 350775:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11842.545125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11842.557555] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11842.647179] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11848.564521] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11860.552845] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11865.901257] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11866.098687] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11871.477496] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11871.737158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11878.371965] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11878.373081] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11878.385379] LustreError: Skipped 1 previous similar message [11878.389937] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11878.398991] Lustre: Skipped 1 previous similar message [11878.401016] Lustre: Skipped 3 previous similar messages [11879.769817] Lustre: server umount lustre-OST0000 complete [11883.491030] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11883.518033] Lustre: Skipped 2 previous similar messages [11889.593959] Lustre: server umount lustre-MDT0000 complete [11893.733126] LustreError: 348816:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11893.746842] LustreError: 348816:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11894.294074] LustreError: 348799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591491 with bad export cookie 14224557661310152798 [11894.296160] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11894.312778] LustreError: 348799:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11894.546418] Lustre: server umount lustre-MDT0001 complete [11903.667979] XFS (loop0): Unmounting Filesystem [11907.991090] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:11:42 (1787591502) [11913.485129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11913.957566] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11913.966841] Lustre: Skipped 2 previous similar messages [11917.932841] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11925.147863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11929.070987] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11936.218260] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11941.881916] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11948.909814] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11955.620828] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [11963.109979] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11971.045485] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11971.062148] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11971.077914] Lustre: Skipped 2 previous similar messages [11971.084566] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11971.089146] Lustre: Skipped 2 previous similar messages [11976.689993] Lustre: server umount lustre-OST0000 complete [11981.793917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11981.800517] LustreError: Skipped 1 previous similar message [11986.549510] Lustre: server umount lustre-MDT0000 complete [11989.473864] LustreError: 355520:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11989.490912] LustreError: 355520:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [11990.095476] LustreError: 354070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787591586 with bad export cookie 14224557661310153680 [11990.098737] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11990.116114] LustreError: 354070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11990.391508] Lustre: server umount lustre-MDT0001 complete [12001.339635] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [12004.827538] Key type lgssc unregistered [12005.230213] LNet: 356965:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12005.243965] LNetError: 356965:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12005.256528] LNet: Removed LNI 192.168.206.126@tcp [12006.096257] Key type .llcrypt unregistered [12006.099657] Key type ._llcrypt unregistered [12031.955684] Key type ._llcrypt registered [12031.956933] Key type .llcrypt registered [12032.029499] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [12032.984386] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12033.043472] alg: No test for adler32 (adler32-zlib) [12034.208637] Lustre: Lustre: Build Version: 2.17.57_80_g3abb7b2 [12034.510292] LNet: Added LNI 192.168.206.126@tcp [8/256/0/180] [12036.232214] Key type lgssc registered [12037.519171] Lustre: Echo OBD driver; http://www.lustre.org/ [12044.218535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12044.238318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12045.606481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12049.755518] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12056.034752] LustreError: 358530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12056.064076] LustreError: 358530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12059.469433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12059.810502] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12064.230093] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12071.397580] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12076.356431] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12083.332312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12083.637962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12089.467849] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12097.354989] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12106.210870] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12106.212394] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12106.234389] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12106.283346] Lustre: Skipped 1 previous similar message [12109.765432] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.26@tcp (stopping) [12109.772202] Lustre: Skipped 1 previous similar message [12111.334750] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12111.339045] Lustre: Skipped 1 previous similar message [12111.874563] Lustre: server umount lustre-MDT0001 complete [12114.855137] LustreError: 358525:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12116.468260] LustreError: 358530:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12116.497120] LustreError: 358530:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12119.983281] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12120.694080] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12120.790930] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12120.799557] Lustre: Found index 1 for lustre-MDT0001, updating log [12120.866051] Lustre: 358519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12121.194027] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12125.233158] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12126.695537] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12130.795693] LustreError: 362132:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12130.799447] LustreError: 362132:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12130.805254] LustreError: 358522:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12130.849852] LustreError: 362135:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.126@tcp: cfg command failed: rc = -17 [12130.856887] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12130.856887] [12432.326493] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.26@tcp (stopping) [12433.890534] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12433.896374] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12436.650454] Lustre: server umount lustre-MDT0001 complete [12437.423338] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12447.429910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12447.500501] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12447.507097] Lustre: Found index 1 for lustre-MDT0001, updating log [12447.538334] Lustre: 358520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12447.659273] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12447.681468] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12447.909300] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12452.346155] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12453.356979] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12453.403330] LustreError: 370807:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12453.418077] LustreError: 370807:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12453.431200] LustreError: 358522:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12453.439397] LustreError: 358522:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12453.505878] LustreError: 370816:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12453.513351] LustreError: 370816:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12453.517396] LustreError: 370816:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.126@tcp: cfg command failed: rc = -17 [12453.528133] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.126@tcp [12453.528133] [12760.024769] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.26@tcp (stopping) [12760.029841] Lustre: Skipped 2 previous similar messages [12760.552813] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12760.556434] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12760.577660] Lustre: Skipped 1 previous similar message [12763.985265] Lustre: server umount lustre-MDT0001 complete [12765.092586] LustreError: 358527:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.26@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12775.181120] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12775.359341] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12775.371903] Lustre: Found index 1 for lustre-MDT0001, updating log [12775.469079] Lustre: 358520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12775.930673] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12780.693635] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [12781.036020] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12781.068322] LustreError: 379598:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12781.083596] LustreError: 379598:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12781.107865] LustreError: 358522:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12781.111352] LustreError: 358522:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12781.212693] LustreError: 379602:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12781.238409] LustreError: 379602:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.126@tcp: cfg command failed: rc = -114 [12781.261408] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12781.261408] [13103.586607] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13103.592818] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13103.598809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13103.609348] Lustre: Skipped 3 previous similar messages [13103.615905] Lustre: Skipped 3 previous similar messages [13108.066873] Lustre: server umount lustre-MDT0000 complete [13108.718322] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13108.729169] LustreError: 358526:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [13111.795605] LustreError: 358512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787592708 with bad export cookie 148430053255159194 [13111.804956] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13111.823956] LustreError: 358512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13113.843915] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13118.466560] Lustre: server umount lustre-MDT0001 complete [13123.776607] Lustre: server umount lustre-OST0000 complete [13135.571694] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [13147.956702] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [13185.345660] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 13:32:59 (1787592779) [13192.773805] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13193.154087] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13193.196605] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13193.286660] Lustre: lustre-MDT0000: new disk, initializing [13193.387453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13193.431422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13199.634870] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [13211.519410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13211.680279] Lustre: 390513:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13211.764738] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13211.768633] Lustre: Skipped 1 previous similar message [13211.857591] Lustre: lustre-MDT0001: new disk, initializing [13211.955305] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13212.005509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13212.026415] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13216.964715] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [13226.970630] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13234.280912] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13243.367536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13243.964083] Lustre: lustre-OST0000: new disk, initializing [13243.970622] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13243.980757] Lustre: 392485:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13244.095057] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13252.042059] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_default_debug -1 all [13253.243796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13253.273511] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13253.395134] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13263.381658] Lustre: DEBUG MARKER: oleg626-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13272.244258] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13272.249305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13272.272304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13272.280391] Lustre: Skipped 2 previous similar messages [13278.448835] Lustre: server umount lustre-MDT0000 complete [13279.206430] LustreError: 392431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13279.227680] LustreError: 392431:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13282.058832] LustreError: 390506:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787592878 with bad export cookie 148430053255161154 [13282.073480] LustreError: 391320:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13282.079135] LustreError: MGC192.168.206.126@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13282.314616] Lustre: server umount lustre-MDT0001 complete [13297.275642] Lustre: server umount lustre-OST0000 complete [13325.909956] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing set_hostid [13337.434438] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing load_modules_local [13378.292626] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13101 sec ========= 13:36:12 (1787592972) [13380.528467] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 13:36:14 (1787592974) === [13384.830211] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 13:36:19 (1787592979) === [13426.440663] Lustre: DEBUG MARKER: oleg626-server.virtnet: executing unload_modules_local [13429.050420] Key type lgssc unregistered [13429.370883] LNet: 398857:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13429.384430] LNetError: 398857:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13429.398807] LNet: Removed LNI 192.168.206.126@tcp [13430.475445] Key type .llcrypt unregistered [13430.478430] Key type ._llcrypt unregistered