[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 555816612 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003308] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008737] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010016] pid_max: default: 32768 minimum: 301 [ 0.011158] LSM: Security Framework initializing [ 0.012060] Yama: becoming mindful. [ 0.013038] SELinux: Initializing. [ 0.014070] *** VALIDATE selinux *** [ 0.022482] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027549] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028154] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029128] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030119] *** VALIDATE tmpfs *** [ 0.031479] *** VALIDATE proc *** [ 0.032231] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034273] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035153] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037031] Spectre V2 : User space: Vulnerable [ 0.038010] Speculative Store Bypass: Vulnerable [ 0.041275] debug: unmapping init [mem 0xffffffffa0a59000-0xffffffffa0a60fff] [ 0.043203] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044724] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045023] ... version: 2 [ 0.046012] ... bit width: 48 [ 0.047012] ... generic registers: 4 [ 0.048012] ... value mask: 0000ffffffffffff [ 0.049014] ... max period: 00007fffffffffff [ 0.050016] ... fixed-purpose events: 3 [ 0.051015] ... event mask: 000000070000000f [ 0.052289] rcu: Hierarchical SRCU implementation. [ 0.054451] smp: Bringing up secondary CPUs ... [ 0.055587] x86: Booting SMP configuration: [ 0.056025] .... node #0, CPUs: #1 #2 #3 [ 0.060158] smp: Brought up 1 node, 4 CPUs [ 0.062086] smpboot: Max logical packages: 1 [ 0.063015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.134458] node 0 deferred pages initialised in 69ms [ 0.138088] devtmpfs: initialized [ 0.140326] x86/mm: Memory block size: 128MB [ 0.142938] gcov: version magic: 0x41383552 [ 0.145333] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.146100] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.147270] pinctrl core: initialized pinctrl subsystem [ 0.148248] [ 0.149011] ************************************************************* [ 0.150014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.151016] ** ** [ 0.152012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.153015] ** ** [ 0.154026] ** This means that this kernel is built to expose internal ** [ 0.155013] ** IOMMU data structures, which may compromise security on ** [ 0.156013] ** your system. ** [ 0.157013] ** ** [ 0.158015] ** If you see this message and you are not debugging the ** [ 0.159011] ** kernel, report this immediately to your vendor! ** [ 0.160012] ** ** [ 0.161010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162011] ************************************************************* [ 0.163658] NET: Registered protocol family 16 [ 0.164615] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.165057] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.166058] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.167602] cpuidle: using governor menu [ 0.169599] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.172639] PCI: Using configuration type 1 for base access [ 0.174137] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.183106] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.185131] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.189234] cryptd: max_cpu_qlen set to 1000 [ 0.192200] ACPI: Added _OSI(Module Device) [ 0.194016] ACPI: Added _OSI(Processor Device) [ 0.196015] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.197011] ACPI: Added _OSI(Processor Aggregator Device) [ 0.201000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.206565] ACPI: Interpreter enabled [ 0.208060] ACPI: PM: (supports S0 S3 S4 S5) [ 0.210013] ACPI: Using IOAPIC for interrupt routing [ 0.212136] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.216613] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.232283] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.236054] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.239023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.243185] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.249519] acpiphp: Slot [2] registered [ 0.251137] acpiphp: Slot [5] registered [ 0.253093] acpiphp: Slot [6] registered [ 0.256189] acpiphp: Slot [7] registered [ 0.258159] acpiphp: Slot [8] registered [ 0.260188] acpiphp: Slot [9] registered [ 0.263146] acpiphp: Slot [10] registered [ 0.265121] acpiphp: Slot [3] registered [ 0.268121] acpiphp: Slot [4] registered [ 0.269107] acpiphp: Slot [11] registered [ 0.271215] acpiphp: Slot [12] registered [ 0.273110] acpiphp: Slot [13] registered [ 0.275132] acpiphp: Slot [14] registered [ 0.276106] acpiphp: Slot [15] registered [ 0.278110] acpiphp: Slot [16] registered [ 0.279094] acpiphp: Slot [17] registered [ 0.281118] acpiphp: Slot [18] registered [ 0.283166] acpiphp: Slot [19] registered [ 0.284087] acpiphp: Slot [20] registered [ 0.286091] acpiphp: Slot [21] registered [ 0.287117] acpiphp: Slot [22] registered [ 0.289072] acpiphp: Slot [23] registered [ 0.291110] acpiphp: Slot [24] registered [ 0.294139] acpiphp: Slot [25] registered [ 0.296121] acpiphp: Slot [26] registered [ 0.298119] acpiphp: Slot [27] registered [ 0.301130] acpiphp: Slot [28] registered [ 0.304284] acpiphp: Slot [29] registered [ 0.305093] acpiphp: Slot [30] registered [ 0.307137] acpiphp: Slot [31] registered [ 0.308059] PCI host bridge to bus 0000:00 [ 0.310016] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.312024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.313029] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.315021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.318020] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.320028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.322187] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.325998] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.329199] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.338017] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.342011] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.344023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.346015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.348015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.351332] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.355831] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.360052] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.365907] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.372017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.385014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.389011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.397166] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.407018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.420019] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.448019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.463158] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.469018] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.479017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.509015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.522323] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.532017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.539018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.559016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.569000] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.580018] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.590030] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.613024] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.627541] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.635024] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.646025] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.674022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.690889] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.698021] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.707024] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.730027] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.744663] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.748685] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.751526] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.753469] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.756373] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.763057] iommu: Default domain type: Passthrough [ 0.765507] SCSI subsystem initialized [ 0.766157] ACPI: bus type USB registered [ 0.768120] usbcore: registered new interface driver usbfs [ 0.770156] usbcore: registered new interface driver hub [ 0.772185] usbcore: registered new device driver usb [ 0.774214] pps_core: LinuxPPS API ver. 1 registered [ 0.776013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.779083] PTP clock support registered [ 0.781181] EDAC MC: Ver: 3.0.0 [ 0.783167] PCI: Using ACPI for IRQ routing [ 0.784961] NetLabel: Initializing [ 0.786013] NetLabel: domain hash size = 128 [ 0.788012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.790101] NetLabel: unlabeled traffic allowed by default [ 0.792133] vgaarb: loaded [ 0.793335] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.795021] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.802000] clocksource: Switched to clocksource kvm-clock [ 0.918086] VFS: Disk quotas dquot_6.6.0 [ 0.919634] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.922480] *** VALIDATE ramfs *** [ 0.925338] *** VALIDATE hugetlbfs *** [ 0.927288] pnp: PnP ACPI init [ 0.930644] pnp: PnP ACPI: found 6 devices [ 0.955822] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.959500] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.961925] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.964337] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.967294] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.969986] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.973262] NET: Registered protocol family 2 [ 0.976045] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.981409] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.985468] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.991352] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.995432] TCP: Hash tables configured (established 65536 bind 65536) [ 0.998980] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.002443] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.005592] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.009019] NET: Registered protocol family 1 [ 1.012309] RPC: Registered named UNIX socket transport module. [ 1.015042] RPC: Registered udp transport module. [ 1.017926] RPC: Registered tcp transport module. [ 1.020766] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.024218] NET: Registered protocol family 44 [ 1.026151] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.028500] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.030823] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.035144] PCI: CLS 0 bytes, default 64 [ 1.037236] Unpacking initramfs... [ 2.506302] debug: unmapping init [mem 0xffff9b933cc54000-0xffff9b933ffbffff] [ 2.513169] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.515635] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.520181] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.034450] Initialise system trusted keyrings [ 3.036868] Key type blacklist registered [ 3.038915] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.047952] zbud: loaded [ 3.051243] *** VALIDATE nfs *** [ 3.052596] *** VALIDATE nfs4 *** [ 3.054571] pstore: using deflate compression [ 3.059629] Platform Keyring initialized [ 3.168655] NET: Registered protocol family 38 [ 3.171666] Key type asymmetric registered [ 3.173877] Asymmetric key parser 'x509' registered [ 3.176387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.180849] io scheduler mq-deadline registered [ 3.183321] io scheduler kyber registered [ 3.185196] io scheduler bfq registered [ 3.187390] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.190862] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.194253] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.197753] ACPI: Power Button [PWRF] [ 3.203939] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.211649] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.229493] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.240365] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.255263] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.285280] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.312715] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.320121] Non-volatile memory driver v1.3 [ 3.322019] Linux agpgart interface v0.103 [ 3.354337] virtio_blk virtio1: [vda] 146144 512-byte logical blocks (74.8 MB/71.4 MiB) [ 3.357121] vda: detected capacity change from 0 to 74825728 [ 3.376753] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.380332] vdb: detected capacity change from 0 to 1073741824 [ 3.405141] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.408280] vdc: detected capacity change from 0 to 2621440000 [ 3.423816] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.426556] vdd: detected capacity change from 0 to 2621440000 [ 3.443207] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.446472] vde: detected capacity change from 0 to 4294967296 [ 3.464920] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.467941] vdf: detected capacity change from 0 to 4294967296 [ 3.496371] libphy: Fixed MDIO Bus: probed [ 3.526726] usbcore: registered new interface driver usbserial_generic [ 3.530843] usbserial: USB Serial support registered for generic [ 3.534713] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.541530] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.543620] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.547167] mousedev: PS/2 mouse device common for all mice [ 3.554517] rtc_cmos 00:05: RTC can wake from S4 [ 3.558370] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.561304] rtc_cmos 00:05: registered as rtc0 [ 3.567593] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.568308] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.572569] intel_pstate: CPU model not supported [ 3.578969] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.581488] hid: raw HID events driver (C) Jiri Kosina [ 3.586586] usbcore: registered new interface driver usbhid [ 3.589081] usbhid: USB HID core driver [ 3.590719] drop_monitor: Initializing network drop monitor service [ 3.593248] Initializing XFRM netlink socket [ 3.595576] NET: Registered protocol family 10 [ 3.598292] Segment Routing with IPv6 [ 3.599984] NET: Registered protocol family 17 [ 3.602432] mpls_gso: MPLS GSO support [ 3.610564] RAS: Correctable Errors collector initialized. [ 3.613457] AVX version of gcm_enc/dec engaged. [ 3.615796] AES CTR mode by8 optimization enabled [ 3.725859] sched_clock: Marking stable (3725825784, 0)->(4712782440, -986956656) [ 3.729820] registered taskstats version 1 [ 3.732343] Loading compiled-in X.509 certificates [ 3.734774] zswap: loaded using pool lzo/zbud [ 3.764248] Key type big_key registered [ 3.779302] Key type encrypted registered [ 3.781289] ima: No TPM chip found, activating TPM-bypass! [ 3.783222] ima: Allocated hash algorithm: sha1 [ 3.785157] ima: No architecture policies found [ 3.787556] evm: Initialising EVM extended attributes: [ 3.789902] evm: security.selinux [ 3.791425] evm: security.ima [ 3.792948] evm: security.capability [ 3.794410] evm: HMAC attrs: 0x1 [ 3.797258] rtc_cmos 00:05: setting system clock to 2026-08-25 22:31:48 UTC (1787697108) [ 3.804426] debug: unmapping init [mem 0xffffffffa1a03000-0xffffffffa1bfffff] [ 3.807522] debug: unmapping init [mem 0xffffffffa0782000-0xffffffffa0a58fff] [ 3.817094] Write protecting the kernel read-only data: 28672k [ 3.821089] debug: unmapping init [mem 0xffffffff9ee03000-0xffffffff9effffff] [ 3.824248] debug: unmapping init [mem 0xffffffff9f714000-0xffffffff9f7fffff] [ 3.865979] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.876329] systemd[1]: Detected virtualization kvm. [ 3.878470] systemd[1]: Detected architecture x86-64. [ 3.881079] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.913845] systemd[1]: No hostname configured. [ 3.915956] systemd[1]: Set hostname to . [ 3.918260] random: systemd: uninitialized urandom read (16 bytes read) [ 3.921269] systemd[1]: Initializing machine ID from random generator. [ 4.078539] random: systemd: uninitialized urandom read (16 bytes read) [ 4.081186] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.086757] random: systemd: uninitialized urandom read (16 bytes read) [ 4.089430] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.093977] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Sockets. [ OK ] Reached target Initrd Root Device. Starting Setup Virtual Console... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.789489] device-mapper: uevent: version 1.0.3 [ 4.791543] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.532749] virtio_net virtio0 ens2: renamed from eth0 [ 5.585388] random: fast init done [ 5.642673] scsi host0: ata_piix [ 5.651531] scsi host1: ata_piix [ 5.652909] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.655751] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 10.133970] dracut-initqueue[583]: RTNETLINK answers: File exists [ 10.302904] random: crng init done [ 10.306495] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.882947] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.171726] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.442353] SELinux: Disabled at runtime. [ 12.507791] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.518546] systemd[1]: Detected virtualization kvm. [ 12.521114] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.077433] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.081110] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.085714] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.090336] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.094208] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.104720] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.115828] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on udev Kernel Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ 13.226965] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-getty.slice. Starting udev Coldplug all Devices... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.602050] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.954931] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.011276] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.113734] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.141508] EDAC sbridge: Ver: 1.1.2 [ 15.912898] Key type dns_resolver registered [ 16.249754] NFS: Registering the id_resolver key type [ 16.252137] Key type id_resolver registered [ 16.254187] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started RPC Bind. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg449-server login: [ 43.440305] libcfs: loading out-of-tree module taints kernel. [ 43.461783] Key type ._llcrypt registered [ 43.463495] Key type .llcrypt registered [ 43.524450] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 65.490564] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 67.185630] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 67.199886] alg: No test for adler32 (adler32-zlib) [ 68.629635] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 69.473797] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 71.231426] Key type lgssc registered [ 74.250177] Lustre: Echo OBD driver; http://www.lustre.org/ [ 81.051070] hrtimer: interrupt took 6061144 ns [ 99.023339] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 142.860533] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 156.774776] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 156.819584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 158.290695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 158.335231] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 158.538620] Lustre: lustre-MDT0000: new disk, initializing [ 158.691080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 158.733560] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 164.251686] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 178.785818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 178.938893] Lustre: 6506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 178.983950] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 178.987205] Lustre: Skipped 1 previous similar message [ 179.073508] Lustre: lustre-MDT0001: new disk, initializing [ 179.143486] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 179.185810] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 179.196375] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 184.689237] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 190.983545] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 207.978224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 208.820391] Lustre: lustre-OST0000: new disk, initializing [ 208.836833] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 208.855485] Lustre: 8442:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 209.022117] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 216.651186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 216.668653] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 216.823198] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 219.161248] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 234.190643] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 234.338383] Lustre: lustre-OST0001: new disk, initializing [ 234.341911] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 234.347469] Lustre: 9516:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 234.410696] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 241.693731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 241.700850] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 241.760972] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 242.050801] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 254.734850] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 263.289134] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 270.079656] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing check_logdir /tmp/testlogs/ [ 275.242504] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing yml_node [ 280.246560] Lustre: DEBUG MARKER: Client: 2.17.57.80 [ 283.157908] Lustre: DEBUG MARKER: MDS: 2.17.57.80 [ 286.123554] Lustre: DEBUG MARKER: OSS: 2.17.57.80 [ 288.127502] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Aug 25 18:36:30 EDT 2026 [ 307.440484] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 308.953698] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 327.648030] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 327.655523] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 327.680892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.675031] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.677542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.719423] Lustre: Skipped 3 previous similar messages [ 331.668143] Lustre: server umount lustre-MDT0000 complete [ 336.104939] LustreError: 6496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787697440 with bad export cookie 15599069613247198733 [ 336.106654] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 336.126969] LustreError: 6496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 338.913533] LustreError: 7929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 338.915262] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 338.947368] LustreError: 7929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 338.996884] Lustre: Skipped 2 previous similar messages [ 339.003970] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 339.016731] Lustre: Skipped 1 previous similar message [ 341.920985] LustreError: 6510:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 341.925951] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 341.945677] LustreError: 6510:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 341.962931] Lustre: Skipped 1 previous similar message [ 342.708076] Lustre: server umount lustre-MDT0001 complete [ 347.341620] Lustre: server umount lustre-OST0000 complete [ 351.725407] Lustre: server umount lustre-OST0001 complete [ 359.315620] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 368.228829] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 410.077095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 410.493064] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 410.536984] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 410.621295] Lustre: lustre-MDT0000: new disk, initializing [ 410.724984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.740127] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 416.358595] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 430.430166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 430.648987] Lustre: 15073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 430.762098] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 430.770720] Lustre: Skipped 1 previous similar message [ 430.857881] Lustre: lustre-MDT0001: new disk, initializing [ 430.991484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 431.092218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 431.113327] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 436.157498] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 446.750320] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 453.633264] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 460.959348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 461.370336] Lustre: lustre-OST0000: new disk, initializing [ 461.374691] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 461.383398] Lustre: 17046:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 461.456474] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 462.594719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 462.607292] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 462.737296] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 469.112360] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 479.577249] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 485.153922] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 485.464403] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 490.910638] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 491.096708] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 493.544276] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 493.549629] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 493.560679] LustreError: Skipped 1 previous similar message [ 493.563103] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 493.595670] Lustre: Skipped 2 previous similar messages [ 503.784897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 503.796800] Lustre: Skipped 4 previous similar messages [ 507.359257] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 507.633455] Lustre: server umount lustre-OST0000 complete [ 512.994332] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 513.011317] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 517.848928] Lustre: server umount lustre-MDT0000 complete [ 519.144737] LustreError: 15083:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.182528] LustreError: 15083:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 522.136395] LustreError: 15868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787697626 with bad export cookie 15599069613247200266 [ 522.147373] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 522.151720] LustreError: 15868:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 522.490572] Lustre: server umount lustre-MDT0001 complete [ 528.271620] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:40:31 (1787697631) [ 535.370988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 535.792597] LustreError: 19313:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 535.883128] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 540.839294] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 541.160900] LustreError: 19314:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 550.517127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 556.369508] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 564.588346] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 573.133490] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 584.501653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 593.973859] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 603.268753] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 612.508391] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 621.541752] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 621.546609] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 621.568169] Lustre: Skipped 3 previous similar messages [ 621.580442] Lustre: Skipped 5 previous similar messages [ 626.579516] Lustre: server umount lustre-MDT0000 complete [ 626.662928] LustreError: 20041:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 626.693333] LustreError: 20041:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 631.449382] LustreError: 21157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787697736 with bad export cookie 15599069613247201148 [ 631.450628] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 631.477470] LustreError: 21157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 632.144738] Lustre: server umount lustre-MDT0001 complete [ 638.749206] Lustre: server umount lustre-OST0000 complete [ 663.446366] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 673.568045] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 674.023132] LustreError: 23694:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 674.044441] LustreError: 23694:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 674.122992] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 674.136512] Lustre: Skipped 2 previous similar messages [ 678.921846] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 687.709302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 693.176461] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 696.211295] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 703.329460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 711.222328] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 720.655155] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 720.883479] Lustre: lustre-OST0001: new disk, initializing [ 720.888641] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 720.904429] Lustre: 26015:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 722.285657] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 722.303909] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 722.472181] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 728.105819] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 739.206712] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 744.607058] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 761.316835] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 761.321724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 761.340903] Lustre: Skipped 3 previous similar messages [ 761.362905] Lustre: Skipped 3 previous similar messages [ 766.450324] LustreError: 25453:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 766.474688] LustreError: 25453:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 766.659783] Lustre: server umount lustre-MDT0000 complete [ 771.008603] LustreError: 23675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787697875 with bad export cookie 15599069613247201995 [ 771.013115] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 771.017167] LustreError: 23675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 771.424914] Lustre: server umount lustre-MDT0001 complete [ 785.883539] Lustre: server umount lustre-OST0000 complete [ 787.935153] Lustre: 3636:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787697876/real 1787697876] req@ffff9b93bdc1e300 x1874536352182272/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787697892 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 789.786250] Lustre: server umount lustre-OST0001 complete [ 806.638917] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 817.389761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 817.858401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 817.862272] Lustre: Skipped 3 previous similar messages [ 823.166026] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 832.278965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 832.483803] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 838.356450] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 841.447721] Lustre: 30178:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 849.990312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 857.901240] LustreError: 30532:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 857.917200] LustreError: 30532:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 858.128228] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 867.298270] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 874.118887] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 882.134750] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 886.189686] Lustre: 32058:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 893.408819] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 893.440051] Lustre: Skipped 5 previous similar messages [ 899.189511] Lustre: server umount lustre-MDT0000 complete [ 903.215416] LustreError: 32060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787698007 with bad export cookie 15599069613247203290 [ 903.221048] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 903.227424] LustreError: 32060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 903.654781] Lustre: server umount lustre-MDT0001 complete [ 918.094227] Lustre: server umount lustre-OST0000 complete [ 920.032897] Lustre: 3638:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787698008/real 1787698008] req@ffff9b938732ed80 x1874536352231808/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787698024 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 922.229062] Lustre: server umount lustre-OST0001 complete [ 931.023575] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:47:13 (1787698033) [ 963.524681] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 976.420357] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 982.608171] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 987.113492] LustreError: 34863:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 987.146961] LustreError: 34863:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 992.491798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 997.160668] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1000.841950] Lustre: 36012:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1008.159297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1015.284860] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1025.649883] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1033.294512] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1041.928825] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1046.444697] Lustre: 37892:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1054.177455] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1054.185551] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1054.199260] Lustre: Skipped 1 previous similar message [ 1054.204842] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1054.211086] Lustre: Skipped 8 previous similar messages [ 1059.124919] Lustre: server umount lustre-MDT0000 complete [ 1063.961961] LustreError: 34843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787698168 with bad export cookie 15599069613247204396 [ 1063.991691] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1064.513928] Lustre: server umount lustre-MDT0001 complete [ 1078.458287] Lustre: server umount lustre-OST0000 complete [ 1082.335122] Lustre: 3636:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787698171/real 1787698171] req@ffff9b9285093800 x1874536352283392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787698187 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1083.338723] Lustre: server umount lustre-OST0001 complete [ 1093.734668] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:49:56 (1787698196) [ 1111.257654] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1121.842098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.347025] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1122.353504] Lustre: Skipped 7 previous similar messages [ 1126.824265] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1135.973886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1141.119716] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1144.345884] Lustre: 41182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1151.824083] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1159.077590] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1168.634428] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1176.107867] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1183.814735] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1188.955495] Lustre: 43060:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1189.902498] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1189.909871] Lustre: Skipped 1 previous similar message [ 1194.468914] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1194.486457] Lustre: Skipped 6 previous similar messages [ 1200.020577] Lustre: server umount lustre-MDT0000 complete [ 1203.993697] LustreError: 41184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787698308 with bad export cookie 15599069613247205502 [ 1204.005192] LustreError: 41184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1204.013057] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1210.452947] Lustre: server umount lustre-MDT0001 complete [ 1220.996771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1234.720880] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1239.041617] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1244.145659] LustreError: 43935:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1244.183351] LustreError: 43935:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 31 previous similar messages [ 1244.775184] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1249.893295] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1250.793972] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1250.798952] Lustre: Skipped 1 previous similar message [ 1258.195770] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1259.889796] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1266.604979] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1268.601944] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1275.878322] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1280.370384] Lustre: server umount lustre-MDT0000 complete [ 1283.908751] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787698388 with bad export cookie 15599069613247206594 [ 1283.922702] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1283.930673] LustreError: 40013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1284.434530] Lustre: server umount lustre-MDT0001 complete [ 1299.256228] Lustre: server umount lustre-OST0000 complete [ 1302.751155] Lustre: 3635:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787698391/real 1787698391] req@ffff9b93b7b6b800 x1874536352370944/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787698407 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1304.766534] Lustre: server umount lustre-OST0001 complete [ 1316.456741] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:53:39 (1787698419) [ 1335.520129] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1345.712957] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1350.941556] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1359.836622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1365.007537] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1368.569447] Lustre: 48825:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1376.833058] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1384.418763] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1394.020972] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1400.602550] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1408.131221] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1412.169377] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1424.327370] Lustre: server umount lustre-MDT0000 complete [ 1428.423458] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787698533 with bad export cookie 15599069613247207413 [ 1428.441754] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1428.445239] LustreError: 47652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1428.688645] Lustre: server umount lustre-MDT0001 complete [ 1442.487282] Lustre: server umount lustre-OST0000 complete [ 1445.352806] Lustre: 3638:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787698534/real 1787698534] req@ffff9b92829f1880 x1874536352426880/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787698550 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1447.072156] Lustre: server umount lustre-OST0001 complete [ 1459.422112] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:56:01 (1787698561) [ 1463.083518] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1466.004707] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:56:07 (1787698567) [ 1480.437357] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 1490.397048] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1513.163308] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1513.487657] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1513.605516] Lustre: lustre-MDT0000: new disk, initializing [ 1513.715312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1519.032349] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1530.226299] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1538.721858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1539.014869] Lustre: lustre-OST0000: new disk, initializing [ 1539.018703] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1539.027288] Lustre: Skipped 1 previous similar message [ 1539.035918] Lustre: 54902:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1540.490900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1540.503931] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1540.539425] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1546.975300] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1558.461183] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1560.912606] Lustre: DEBUG MARKER: create test files [ 1596.907968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1596.910950] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1596.925505] Lustre: Skipped 12 previous similar messages [ 1596.950677] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1596.963231] Lustre: Skipped 25 previous similar messages [ 1602.514549] Lustre: server umount lustre-OST0000 complete [ 1607.578629] Lustre: server umount lustre-MDT0000 complete [ 1620.413650] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 1623.270729] Key type lgssc unregistered [ 1623.570225] LNet: 56540:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1623.592111] LNetError: 56540:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1623.614740] LNet: Removed LNI 192.168.204.149@tcp [ 1624.584187] Key type .llcrypt unregistered [ 1624.589087] Key type ._llcrypt unregistered [ 1654.406309] Key type ._llcrypt registered [ 1654.408337] Key type .llcrypt registered [ 1654.564122] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1656.396150] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1656.478266] alg: No test for adler32 (adler32-zlib) [ 1657.493505] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 1657.933674] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 1659.775219] Key type lgssc registered [ 1661.176779] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1672.341752] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1672.392391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1674.216590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1679.571436] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1687.754693] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1695.514181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1696.271602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1698.382371] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1698.520875] Lustre: 59278:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1698.536988] Lustre: 59278:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1698.549758] Lustre: 59278:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1698.559029] Lustre: 59278:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1698.571623] Lustre: 59278:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1698.583066] Lustre: 59278:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1702.667778] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1711.882302] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1717.377526] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1723.744191] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1733.085320] Lustre: server umount lustre-OST0000 complete [ 1747.120680] Lustre: server umount lustre-MDT0000 complete [ 1759.603136] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 1762.538334] Key type lgssc unregistered [ 1762.855221] LNet: 60905:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1762.866329] LNetError: 60905:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1762.892332] LNet: Removed LNI 192.168.204.149@tcp [ 1763.923135] Key type .llcrypt unregistered [ 1763.925533] Key type ._llcrypt unregistered [ 1796.513467] Key type ._llcrypt registered [ 1796.516647] Key type .llcrypt registered [ 1796.638091] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1798.007205] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1798.237823] alg: No test for adler32 (adler32-zlib) [ 1799.373361] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 1799.586210] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 1801.303173] Key type lgssc registered [ 1802.473752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1813.187000] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1813.220951] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1815.250856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1821.001882] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1831.467633] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1839.920879] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1840.335287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1845.760509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1845.879225] Lustre: 63747:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1845.891372] Lustre: 63747:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1845.914023] Lustre: 63747:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1845.937169] Lustre: 63747:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1845.953310] Lustre: 63747:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1845.965040] Lustre: 63747:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1848.313329] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 1857.203507] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1862.007726] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1871.329884] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1871.343660] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1871.367886] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1876.451923] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1877.506593] Lustre: server umount lustre-OST0000 complete [ 1881.476532] Lustre: server umount lustre-MDT0000 complete [ 1892.299188] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 1895.268408] Key type lgssc unregistered [ 1895.562120] LNet: 65305:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1895.571110] LNetError: 65305:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1895.586811] LNet: Removed LNI 192.168.204.149@tcp [ 1896.591121] Key type .llcrypt unregistered [ 1896.594615] Key type ._llcrypt unregistered [ 1914.397773] Key type ._llcrypt registered [ 1914.400749] Key type .llcrypt registered [ 1914.521431] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 1931.647368] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 1932.716803] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1932.771838] alg: No test for adler32 (adler32-zlib) [ 1933.863222] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 1934.201201] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 1935.959197] Key type lgssc registered [ 1937.346879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1983.912840] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1983.923371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1985.249358] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1985.340122] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1985.436703] Lustre: lustre-MDT0000: new disk, initializing [ 1985.515926] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1985.582239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1990.508933] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2005.181975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2005.302867] Lustre: 68463:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2005.371125] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2005.382794] Lustre: Skipped 1 previous similar message [ 2005.458541] Lustre: lustre-MDT0001: new disk, initializing [ 2005.599800] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2005.638050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2005.646969] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2010.228669] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2022.274382] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2029.119905] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2037.145133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2037.408383] Lustre: lustre-OST0000: new disk, initializing [ 2037.413160] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2037.418801] Lustre: 70431:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2037.524645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2043.395665] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2046.019652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2046.034396] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2046.152599] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2055.023368] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2060.503867] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2060.768234] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2065.455245] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2065.684511] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2072.037465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2072.048947] Lustre: Skipped 1 previous similar message [ 2072.052776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2077.171159] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2077.180388] Lustre: Skipped 1 previous similar message [ 2082.278760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2082.285760] Lustre: Skipped 1 previous similar message [ 2083.297752] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2083.417888] Lustre: server umount lustre-OST0000 complete [ 2092.512758] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2092.514170] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2092.519859] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2092.526064] Lustre: Skipped 1 previous similar message [ 2092.528900] Lustre: Skipped 1 previous similar message [ 2094.562891] Lustre: server umount lustre-MDT0000 complete [ 2098.005455] LustreError: 69258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787699202 with bad export cookie 5724097942087793256 [ 2098.014135] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2098.019302] LustreError: 69258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2098.339874] Lustre: server umount lustre-MDT0001 complete [ 2107.652950] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:06:50 (1787699210) [ 2117.559959] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 2158.343375] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 2168.726447] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 2210.402722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2210.700448] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2210.769566] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2210.853861] Lustre: lustre-MDT0000: new disk, initializing [ 2210.925738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2210.944975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2215.621969] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2229.151487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2229.266811] Lustre: 76453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2229.316517] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2229.319658] Lustre: Skipped 1 previous similar message [ 2229.410928] Lustre: lustre-MDT0001: new disk, initializing [ 2229.499473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2229.539032] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2229.548718] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2233.452967] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2243.376344] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2249.599070] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2259.358393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2259.617492] Lustre: lustre-OST0000: new disk, initializing [ 2259.621822] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2259.632939] Lustre: 78419:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2259.718914] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2266.184609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2266.198367] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2266.232210] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2266.330810] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2275.532078] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2280.895258] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2281.174757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2285.963474] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2286.177373] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2292.195694] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2292.202295] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2292.218926] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2292.218943] Lustre: Skipped 1 previous similar message [ 2292.237438] LustreError: Skipped 1 previous similar message [ 2294.508748] Lustre: server umount lustre-OST0000 complete [ 2301.415063] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2301.428313] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2301.449447] Lustre: Skipped 1 previous similar message [ 2301.460233] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2301.468943] Lustre: Skipped 1 previous similar message [ 2305.573965] Lustre: server umount lustre-MDT0000 complete [ 2307.555466] LustreError: 76464:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2307.574876] LustreError: 76464:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2309.621306] LustreError: 76444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787699414 with bad export cookie 5724097942087794138 [ 2309.622771] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2309.630568] LustreError: 76444:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2310.002400] Lustre: server umount lustre-MDT0001 complete [ 2318.967929] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:10:21 (1787699421) [ 2326.428498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2327.211221] LustreError: 80829:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2327.321371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2332.128318] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2332.655155] LustreError: 80830:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2336.741516] LustreError: 80829:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2341.875888] LustreError: 80830:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2342.794463] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2347.969045] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2356.159304] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2362.230315] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2369.848406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2370.194042] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2370.201517] Lustre: Skipped 1 previous similar message [ 2376.689149] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2384.425862] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2388.344771] Lustre: 83493:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting b05ff767-e967-48ca-b69d-1de5a6b54fd8 at adminstrative request [ 2393.066903] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2393.086315] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2403.309623] Lustre: MGS: Client b05ff767-e967-48ca-b69d-1de5a6b54fd8 (at 0@lo) reconnecting [ 2403.315360] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4f7014c031b0014c to 0x4f7014c031b003ec [ 2403.346884] Lustre: MGC192.168.204.149@tcp: Connection restored to 0@lo (at 0@lo) [ 2426.576545] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2426.769477] Lustre: lustre-OST0001: new disk, initializing [ 2426.773436] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2426.779366] Lustre: 83891:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2432.600933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2432.607336] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2432.694574] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2433.608962] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2443.062362] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2448.362064] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2448.368190] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2448.379325] Lustre: Skipped 1 previous similar message [ 2448.393534] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2448.407547] Lustre: Skipped 3 previous similar messages [ 2451.794983] Lustre: server umount lustre-OST0001 complete [ 2453.485622] LustreError: 82644:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2453.495719] LustreError: 82644:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2458.610184] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2458.633922] Lustre: Skipped 1 previous similar message [ 2461.952752] Lustre: server umount lustre-OST0000 complete [ 2466.274648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2466.282408] LustreError: Skipped 1 previous similar message [ 2472.065029] Lustre: server umount lustre-MDT0000 complete [ 2473.952413] LustreError: 80825:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2473.971348] LustreError: 80825:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2475.790591] LustreError: 80810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787699580 with bad export cookie 5724097942087795692 [ 2475.798166] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2475.817453] LustreError: 80810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2476.247177] Lustre: server umount lustre-MDT0001 complete [ 2485.567199] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:13:08 (1787699588) [ 2487.242031] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2489.430663] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:13:12 (1787699592) [ 2491.277178] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2493.962775] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:13:16 (1787699596) [ 2496.359903] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2498.649979] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 19:13:21 (1787699601) [ 2500.816452] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2503.133245] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:13:25 (1787699605) [ 2505.095935] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2512.675628] loop: module loaded [ 2512.712922] loop0: detected capacity change from 0 to 4294967296 [ 2512.763422] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2514.871853] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2519.112490] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2524.404266] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2554.215456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2554.276281] LustreError: 88000:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2554.280192] LustreError: 88000:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2554.288792] LustreError: 88000:0:(tgt_mount.c:2582:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2554.296482] LustreError: 88000:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2575.065868] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 2585.564344] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 2625.187978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2625.494193] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2625.536639] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2625.671419] Lustre: lustre-MDT0000: new disk, initializing [ 2625.775190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2625.780469] Lustre: Skipped 1 previous similar message [ 2625.802332] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2630.234355] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2642.428215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2642.589667] Lustre: 90869:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2642.837529] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2648.199658] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2658.577624] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2665.542745] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2673.809547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2674.148947] Lustre: 92829:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2681.185494] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2682.425677] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2682.614669] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2692.238319] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2697.682941] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2697.889179] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2702.887888] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2703.135821] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2708.448073] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2708.456603] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2708.468368] Lustre: Skipped 3 previous similar messages [ 2708.479110] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2708.492187] Lustre: Skipped 6 previous similar messages [ 2711.531321] Lustre: server umount lustre-OST0000 complete [ 2722.180358] Lustre: server umount lustre-MDT0000 complete [ 2723.808715] LustreError: 90880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2723.833541] LustreError: 90880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2726.771670] LustreError: 90860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787699831 with bad export cookie 5724097942087796308 [ 2726.780575] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2726.786312] LustreError: 90860:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2727.239581] Lustre: server umount lustre-MDT0001 complete [ 2739.292473] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:17:21 (1787699841) [ 2747.181916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2752.783679] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2762.777296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2763.251331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2763.258616] Lustre: Skipped 3 previous similar messages [ 2768.854992] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2778.674617] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2785.331903] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2793.087429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2800.541889] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2810.101730] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2816.885125] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2819.309390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 2824.932750] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2825.224346] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2832.464055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2832.622686] Lustre: lustre-OST0001: new disk, initializing [ 2832.625595] Lustre: Skipped 2 previous similar messages [ 2832.628663] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2832.632621] Lustre: Skipped 3 previous similar messages [ 2832.637887] Lustre: 98607:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2838.184579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2838.193878] Lustre: Skipped 2 previous similar messages [ 2838.200676] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2838.339484] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2839.052035] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2850.668432] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2856.223247] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2856.452896] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2861.315887] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2861.640093] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2882.681813] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2910.540590] Lustre: Failing over lustre-MDT0000 [ 2911.166236] Lustre: server umount lustre-MDT0000 complete [ 2911.712865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2911.726122] LustreError: Skipped 1 previous similar message [ 2911.739558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2911.782229] Lustre: Skipped 3 previous similar messages [ 2911.791219] LustreError: 95959:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2911.815972] LustreError: 95959:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2931.679162] Lustre: 66685:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787700020/real 1787700020] req@00000000d3915c9e x1874538306755968/t0(0) o400->MGC192.168.204.149@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787700036 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2931.720635] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2938.883274] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2938.886082] LDISKFS-fs (dm-0): recovery complete [ 2938.893325] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2941.923477] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x4f7014c031b0ce67 [ 2941.950978] Lustre: MGC192.168.204.149@tcp: Connection restored to 0@lo (at 0@lo) [ 2942.290459] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2946.819349] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2946.820161] Lustre: 95228:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2947.562040] Lustre: 95959:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2947.581600] Lustre: 95959:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2947.596659] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2947.698109] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 2947.707246] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2948.039239] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2948.367036] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2948.377045] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2948.695375] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2948.706233] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2948.720282] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2948.731222] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2949.388278] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2949.401987] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2949.751060] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2949.765653] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2950.745539] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2950.775670] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 2951.424421] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2951.451162] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2951.775127] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2951.788331] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 5 previous similar messages [ 2954.953745] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2954.965674] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2955.617757] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2955.647767] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2955.967157] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2955.975276] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2963.056396] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2963.077198] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2963.738277] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2963.762470] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2964.088016] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2964.096992] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2979.221196] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2979.234559] Lustre: 101446:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 49 previous similar messages [ 2979.890236] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2979.905378] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 2980.240154] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2980.249771] LustreError: 101446:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 49 previous similar messages [ 3006.256336] Lustre: 101446:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3006.275427] Lustre: 101446:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3006.286826] Lustre: 101446:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 492c2cea-49a3-49bd-8119-6882db826740@192.168.204.49@tcp [ 3006.299702] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3006.318155] Lustre: 101446:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@0000000054f402d8 x1874538289764480/t0(8589934778) o35->492c2cea-49a3-49bd-8119-6882db826740@192.168.204.49@tcp:551/0 lens 392/0 e 0 to 0 dl 1787700121 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3006.362679] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3006.371422] Lustre: lustre-MDT0000: Denying connection for new client 492c2cea-49a3-49bd-8119-6882db826740 (at 192.168.204.49@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3006.405295] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3006.435773] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3006.454819] Lustre: Skipped 2 previous similar messages [ 3006.457342] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3006.514462] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3006.515158] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3027.956695] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3027.956924] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3027.966382] LustreError: Skipped 1 previous similar message [ 3027.985834] Lustre: Skipped 3 previous similar messages [ 3027.997416] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3028.009133] Lustre: Skipped 3 previous similar messages [ 3040.223179] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3040.361609] Lustre: server umount lustre-OST0000 complete [ 3044.845137] LustreError: 100044:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3044.882135] LustreError: 100044:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 43 previous similar messages [ 3050.949427] Lustre: server umount lustre-OST0001 complete [ 3061.570748] Lustre: server umount lustre-MDT0000 complete [ 3065.906359] LustreError: 95214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787700170 with bad export cookie 5724097942087847527 [ 3065.908696] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3065.924380] LustreError: 95214:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3066.325360] Lustre: server umount lustre-MDT0001 complete [ 3075.584098] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:22:58 (1787700178) [ 3084.205919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3084.220463] Lustre: *** cfs_fail_loc=197, val=0*** [ 3099.872751] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3104.992338] LustreError: 103403:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.149@tcp: failed processing log, type 4: rc = -110 [ 3110.151019] Lustre: *** cfs_fail_loc=197, val=0*** [ 3110.153752] Lustre: Skipped 65 previous similar messages [ 3130.727946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3130.739277] Lustre: Skipped 3 previous similar messages [ 3137.488509] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3146.213074] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3148.442787] Lustre: server umount lustre-OST0000 complete [ 3157.329951] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:24:20 (1787700260) [ 3179.910902] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3185.634775] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3195.788247] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3199.714249] LustreError: 105042:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3199.740022] LustreError: 105042:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3199.748681] LustreError: 105042:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3200.216281] Lustre: server umount lustre-MDT0000 complete [ 3246.206719] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 3259.769818] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 3312.582707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3312.990347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3313.024334] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3313.133130] Lustre: lustre-MDT0000: new disk, initializing [ 3313.235836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3318.111861] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3330.506279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3330.669700] Lustre: 109208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3331.022837] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3336.463095] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3348.918456] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3357.276111] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3365.282442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3365.790381] Lustre: 111167:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3373.316872] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3374.318600] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3383.903663] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3390.086328] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3390.362434] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3394.755922] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3395.145264] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3400.162532] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3400.166385] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3400.166886] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3400.166890] LustreError: Skipped 1 previous similar message [ 3400.179240] Lustre: Skipped 6 previous similar messages [ 3400.221694] Lustre: Skipped 14 previous similar messages [ 3404.153666] Lustre: server umount lustre-OST0000 complete [ 3416.863880] Lustre: server umount lustre-MDT0000 complete [ 3420.649325] LustreError: 109219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3420.660218] LustreError: 109219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 3420.747723] LustreError: 109198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787700525 with bad export cookie 5724097942087858209 [ 3420.787416] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3421.158852] Lustre: server umount lustre-MDT0001 complete [ 3431.570313] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:28:54 (1787700534) [ 3441.208727] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 3444.469331] Key type lgssc unregistered [ 3444.794073] LNet: 113650:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3444.802911] LNetError: 113650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3444.820827] LNet: Removed LNI 192.168.204.149@tcp [ 3445.664849] Key type .llcrypt unregistered [ 3445.668162] Key type ._llcrypt unregistered [ 3464.632189] Key type ._llcrypt registered [ 3464.634706] Key type .llcrypt registered [ 3464.723379] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 3480.682170] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 3481.810408] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3481.911410] alg: No test for adler32 (adler32-zlib) [ 3483.091494] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 3483.466347] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 3485.335182] Key type lgssc registered [ 3486.995603] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3542.521914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3542.562121] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3543.878470] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3543.926952] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3544.008700] Lustre: lustre-MDT0000: new disk, initializing [ 3544.062460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3544.081683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3548.891819] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3560.114495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3560.371924] Lustre: lustre-OST0000: new disk, initializing [ 3560.376671] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3560.381308] Lustre: Skipped 1 previous similar message [ 3560.386501] Lustre: 117812:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3560.453066] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3563.663728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3563.671227] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3563.694784] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3566.800399] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3578.183127] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3578.379302] Lustre: lustre-OST0001: new disk, initializing [ 3578.385225] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3578.397706] Lustre: 118675:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3578.511254] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3585.715576] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3585.731500] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3585.786814] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3586.097946] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3596.257571] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3596.270946] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3596.272378] Lustre: Skipped 1 previous similar message [ 3601.385848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3601.402658] Lustre: Skipped 2 previous similar messages [ 3601.977335] Lustre: server umount lustre-MDT0000 complete [ 3610.628683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3616.288558] LustreError: 117001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787700720 with bad export cookie 12418289653387060928 [ 3616.292760] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3616.299029] LustreError: 117001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3616.558687] Lustre: server umount lustre-OST0000 complete [ 3621.107483] Lustre: server umount lustre-OST0001 complete [ 3631.769720] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 19:32:14 (1787700734) [ 3658.275250] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 19:32:40 (1787700760) [ 3670.865547] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 3698.356111] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:33:20 (1787700800) [ 3743.117807] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 3755.195851] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 3796.038695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3796.241875] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3796.271150] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3796.361971] Lustre: lustre-MDT0000: new disk, initializing [ 3796.461601] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3796.479476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3802.535152] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3815.698368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3815.868440] Lustre: 126403:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3815.970988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3815.979566] Lustre: Skipped 1 previous similar message [ 3816.133517] Lustre: lustre-MDT0001: new disk, initializing [ 3816.319841] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3816.378951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3816.396094] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3822.766978] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3833.916875] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3841.406635] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3850.085135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3850.472967] Lustre: lustre-OST0000: new disk, initializing [ 3850.476302] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3850.482438] Lustre: 128364:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3850.620713] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3858.027647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3858.039694] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3858.280947] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3859.399634] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3871.360437] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3876.814715] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3877.075177] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3882.724726] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3883.006533] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3889.123332] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3889.125052] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3889.137546] LustreError: Skipped 1 previous similar message [ 3889.157056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3889.167737] Lustre: Skipped 1 previous similar message [ 3889.192483] Lustre: Skipped 1 previous similar message [ 3894.243925] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3894.257990] Lustre: Skipped 1 previous similar message [ 3899.365204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3899.369531] Lustre: Skipped 1 previous similar message [ 3899.871146] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3900.206703] Lustre: server umount lustre-OST0000 complete [ 3908.576398] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3908.593256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3908.632875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3911.041287] Lustre: server umount lustre-MDT0000 complete [ 3916.314775] LustreError: 126394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787701021 with bad export cookie 12418289653387062153 [ 3916.329864] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3916.335616] LustreError: 126394:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3916.964702] Lustre: server umount lustre-MDT0001 complete [ 3926.861826] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:37:09 (1787701029) [ 3935.136706] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3935.696565] LustreError: 130768:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3935.718477] LustreError: 130768:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3935.781348] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3940.836460] LustreError: 130769:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3942.315357] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3945.965082] LustreError: 130768:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3951.076804] LustreError: 130769:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3952.830393] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3960.429359] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 3970.951707] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3978.351247] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3985.908217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3986.237319] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3986.248279] Lustre: Skipped 1 previous similar message [ 3993.268301] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4002.015994] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4012.308025] Lustre: *** cfs_fail_loc=159, val=0*** [ 4027.747613] Lustre: lustre-MDT0000: Client 32f4497f-e327-40ef-8715-bf612fc79b43 (at 192.168.204.49@tcp) reconnecting [ 4029.695131] Lustre: *** cfs_fail_loc=159, val=0*** [ 4029.700456] Lustre: Skipped 5 previous similar messages [ 4046.169873] Lustre: lustre-MDT0000: Client 32f4497f-e327-40ef-8715-bf612fc79b43 (at 192.168.204.49@tcp) reconnecting [ 4066.854563] Lustre: server umount lustre-OST0000 complete [ 4072.422575] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4072.424733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4072.446522] Lustre: Skipped 2 previous similar messages [ 4072.466464] Lustre: Skipped 3 previous similar messages [ 4082.655282] Lustre: 115026:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787701171/real 1787701172] req@00000000e1913bd1 x1874539931884672/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787701187 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4082.683608] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4085.727527] Lustre: 115025:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787701174/real 1787701174] req@00000000aff47818 x1874539931885056/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1787701190 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4086.239131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4086.557107] Lustre: server umount lustre-MDT0000 complete [ 4087.785041] LustreError: 132546:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4087.813419] LustreError: 132546:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4088.799345] Lustre: 115027:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787701177/real 1787701177] req@000000003b39f2f0 x1874539931886720/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1787701193 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4091.626856] LustreError: 130749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787701196 with bad export cookie 12418289653387063035 [ 4091.636207] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4091.655086] LustreError: 130749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4092.266990] Lustre: server umount lustre-MDT0001 complete [ 4105.573614] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 4108.596460] Key type lgssc unregistered [ 4108.971347] LNet: 134533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4108.986471] LNetError: 134533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4109.007552] LNet: Removed LNI 192.168.204.149@tcp [ 4110.080514] Key type .llcrypt unregistered [ 4110.086177] Key type ._llcrypt unregistered [ 4138.722672] Key type ._llcrypt registered [ 4138.725833] Key type .llcrypt registered [ 4139.023509] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 4140.656655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4140.704222] alg: No test for adler32 (adler32-zlib) [ 4142.091298] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 4142.469993] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 4144.215819] Key type lgssc registered [ 4145.900728] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4156.390076] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4156.418042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4158.122199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4164.187405] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4167.654185] LustreError: 135941:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4167.692502] LustreError: 135941:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4172.771403] LustreError: 135942:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4174.391492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4174.800645] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4179.512977] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4188.854341] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4195.931334] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4203.807158] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4204.059224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4210.997599] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4219.027501] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4227.655532] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4238.149219] Lustre: *** cfs_fail_loc=159, val=0*** [ 4254.525328] Lustre: lustre-MDT0000: Client 1f01a71b-5836-45b9-a781-05ec39728ad4 (at 192.168.204.49@tcp) reconnecting [ 4257.303900] Lustre: *** cfs_fail_loc=159, val=0*** [ 4272.942823] Lustre: lustre-MDT0001: Client 1f01a71b-5836-45b9-a781-05ec39728ad4 (at 192.168.204.49@tcp) reconnecting [ 4274.903677] Lustre: *** cfs_fail_loc=159, val=0*** [ 4274.911880] Lustre: Skipped 5 previous similar messages [ 4291.408370] Lustre: lustre-MDT0001: Client 1f01a71b-5836-45b9-a781-05ec39728ad4 (at 192.168.204.49@tcp) reconnecting [ 4298.061483] Lustre: 138980:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4301.303371] Lustre: *** cfs_fail_loc=159, val=0*** [ 4301.314532] Lustre: Skipped 2 previous similar messages [ 4317.003871] Lustre: lustre-MDT0000: Client 8d807c34-4a0b-43f7-bec4-d5ccabad3961 (at 192.168.204.49@tcp) reconnecting [ 4318.841961] Lustre: *** cfs_fail_loc=159, val=0*** [ 4318.847633] Lustre: Skipped 14 previous similar messages [ 4335.432672] Lustre: lustre-MDT0000: Client 8d807c34-4a0b-43f7-bec4-d5ccabad3961 (at 192.168.204.49@tcp) reconnecting [ 4337.261863] Lustre: 139273:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4357.600473] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4357.606948] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4357.634420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4360.678717] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4360.684968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4360.685554] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4360.733484] Lustre: Skipped 1 previous similar message [ 4362.287097] Lustre: server umount lustre-OST0000 complete [ 4369.390223] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4369.402759] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4369.421304] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4373.900826] Lustre: server umount lustre-MDT0000 complete [ 4376.032218] LustreError: 139166:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4376.056970] LustreError: 139166:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4378.624203] LustreError: 137777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787701483 with bad export cookie 3881229889303009360 [ 4378.624868] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4378.635849] LustreError: 137777:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4379.094603] Lustre: server umount lustre-MDT0001 complete [ 4393.112905] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 4396.610664] Key type lgssc unregistered [ 4397.167537] LNet: 140355:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4397.175845] LNetError: 140355:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4397.196269] LNet: Removed LNI 192.168.204.149@tcp [ 4398.473283] Key type .llcrypt unregistered [ 4398.476496] Key type ._llcrypt unregistered [ 4424.623878] Key type ._llcrypt registered [ 4424.626460] Key type .llcrypt registered [ 4424.761857] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 4426.345927] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4426.366907] alg: No test for adler32 (adler32-zlib) [ 4427.561851] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 4427.826915] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 4429.555288] Key type lgssc registered [ 4430.797397] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4438.759387] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4438.774044] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4440.214249] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4444.551211] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4449.763138] LustreError: 141763:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4449.780288] LustreError: 141763:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4454.045947] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4454.561125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4459.546356] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4467.238923] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4474.057815] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4480.750848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4481.050492] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4485.174898] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4487.258972] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4495.470116] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4499.508697] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4505.583778] Lustre: 144333:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4509.010931] Lustre: 144381:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4510.777992] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4511.738735] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4515.646232] Lustre: 144528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4516.700264] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4517.574857] Lustre: 144636:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4525.029356] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4525.029719] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4525.043789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4525.048946] Lustre: Skipped 1 previous similar message [ 4525.061691] Lustre: Skipped 1 previous similar message [ 4530.150126] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4530.157775] Lustre: Skipped 1 previous similar message [ 4535.271504] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4535.280168] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4535.301441] Lustre: Skipped 1 previous similar message [ 4535.491529] Lustre: server umount lustre-OST0000 complete [ 4540.388725] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.390432] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4540.410943] Lustre: Skipped 1 previous similar message [ 4540.418635] Lustre: Skipped 1 previous similar message [ 4545.504179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4545.514055] Lustre: Skipped 1 previous similar message [ 4545.915779] Lustre: server umount lustre-MDT0000 complete [ 4550.373309] LustreError: 144431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787701655 with bad export cookie 20909054042727929 [ 4550.375545] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4550.385813] LustreError: 144431:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4550.750304] Lustre: server umount lustre-MDT0001 complete [ 4562.661231] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 4565.524819] Key type lgssc unregistered [ 4565.823769] LNet: 145686:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4565.829091] LNetError: 145686:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4565.849297] LNet: Removed LNI 192.168.204.149@tcp [ 4566.688408] Key type .llcrypt unregistered [ 4566.690411] Key type ._llcrypt unregistered [ 4593.527540] Key type ._llcrypt registered [ 4593.532652] Key type .llcrypt registered [ 4593.686905] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 4594.802182] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4594.829632] alg: No test for adler32 (adler32-zlib) [ 4595.925948] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 4596.248550] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 4597.911208] Key type lgssc registered [ 4599.477928] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4607.639412] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4607.663641] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4609.269032] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4614.319180] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4618.730598] LustreError: 147093:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4618.757191] LustreError: 147093:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4622.393179] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4622.802505] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4627.370544] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4634.682294] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4641.690418] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4649.092419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4649.530969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4653.631742] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4656.325799] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4664.064982] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4667.963105] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4674.064958] Lustre: *** cfs_fail_loc=159, val=0*** [ 4689.754615] Lustre: lustre-MDT0000: Client fe01f7a7-a70f-4e88-9169-4f86e86a5e7a (at 192.168.204.49@tcp) reconnecting [ 4698.599190] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4698.610630] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4698.624802] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4698.631935] Lustre: Skipped 1 previous similar message [ 4702.706328] Lustre: server umount lustre-OST0000 complete [ 4708.837210] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.845436] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4708.856585] Lustre: Skipped 2 previous similar messages [ 4708.881552] Lustre: Skipped 1 previous similar message [ 4713.399874] Lustre: server umount lustre-MDT0000 complete [ 4713.953942] LustreError: 148106:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4713.975031] LustreError: 148106:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4716.885288] LustreError: 147074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787701821 with bad export cookie 14909517770113376019 [ 4716.888820] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4716.906622] LustreError: 147074:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4717.355978] Lustre: server umount lustre-MDT0001 complete [ 4729.977199] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 4732.810547] Key type lgssc unregistered [ 4733.119971] LNet: 150743:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4733.129394] LNetError: 150743:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4733.146751] LNet: Removed LNI 192.168.204.149@tcp [ 4733.967662] Key type .llcrypt unregistered [ 4733.970872] Key type ._llcrypt unregistered [ 4761.389979] Key type ._llcrypt registered [ 4761.396757] Key type .llcrypt registered [ 4761.495104] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 4762.423778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4762.623049] alg: No test for adler32 (adler32-zlib) [ 4763.745656] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 4763.970340] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 4765.671615] Key type lgssc registered [ 4766.859406] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4775.060478] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4775.107884] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4776.605473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4781.546313] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4787.170650] LustreError: 152154:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4787.188405] LustreError: 152154:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4789.710155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4790.079847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4794.051457] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4801.659802] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4808.390859] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4816.685186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4817.153503] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4822.370374] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4825.745507] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 4834.672563] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4835.304199] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4855.972753] Lustre: lustre-MDT0001: evicting 953cf185-d174-4bb1-a666-7248fb0970a6 (at 192.168.204.49@tcp) by administrative request [ 4855.990354] Lustre: Skipped 1 previous similar message [ 4885.984361] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4885.986581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4885.998757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4886.010767] Lustre: Skipped 1 previous similar message [ 4886.019473] Lustre: Skipped 1 previous similar message [ 4891.112326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4891.120232] Lustre: Skipped 1 previous similar message [ 4896.226204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4896.237810] Lustre: Skipped 1 previous similar message [ 4899.297315] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4899.484684] Lustre: server umount lustre-OST0000 complete [ 4906.466434] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4906.477860] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4906.489024] Lustre: Skipped 1 previous similar message [ 4906.507776] Lustre: Skipped 1 previous similar message [ 4909.860604] Lustre: server umount lustre-MDT0000 complete [ 4911.592157] LustreError: 152149:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4911.619677] LustreError: 152149:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4914.069292] LustreError: 152134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787702018 with bad export cookie 11693158185363794861 [ 4914.078102] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4914.094386] LustreError: 152134:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4914.520412] Lustre: server umount lustre-MDT0001 complete [ 4926.935572] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 4930.130939] Key type lgssc unregistered [ 4930.457712] LNet: 156208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4930.466713] LNetError: 156208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4930.492792] LNet: Removed LNI 192.168.204.149@tcp [ 4931.468060] Key type .llcrypt unregistered [ 4931.470347] Key type ._llcrypt unregistered [ 5013.935137] Key type ._llcrypt registered [ 5013.938042] Key type .llcrypt registered [ 5014.067748] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 5015.291115] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5015.320873] alg: No test for adler32 (adler32-zlib) [ 5016.504472] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 5016.829945] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 5018.601459] Key type lgssc registered [ 5019.720760] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5029.200446] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5029.218906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5030.732898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5036.989917] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5041.124212] LustreError: 158684:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5041.163655] LustreError: 158684:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5046.242475] LustreError: 158685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5048.856207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5049.441948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5055.018939] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5064.566264] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5070.858537] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5078.591419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5079.031592] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5085.099589] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5093.376336] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5093.888770] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5100.569516] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5113.832091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5113.838784] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5113.859429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5115.874560] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5115.894556] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5115.897201] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5119.906322] Lustre: server umount lustre-OST0000 complete [ 5126.113452] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5126.147343] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5126.174460] Lustre: Skipped 2 previous similar messages [ 5130.022549] Lustre: server umount lustre-MDT0000 complete [ 5131.246625] LustreError: 158685:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5131.262562] LustreError: 158685:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5133.851667] LustreError: 158665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787702238 with bad export cookie 6375393090127397063 [ 5133.854747] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5133.876140] LustreError: 158665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5134.178117] Lustre: server umount lustre-MDT0001 complete [ 5147.515656] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 5150.542337] Key type lgssc unregistered [ 5150.898607] LNet: 162237:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5150.907609] LNetError: 162237:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5150.930860] LNet: Removed LNI 192.168.204.149@tcp [ 5152.071380] Key type .llcrypt unregistered [ 5152.076050] Key type ._llcrypt unregistered [ 5181.561488] Key type ._llcrypt registered [ 5181.564883] Key type .llcrypt registered [ 5181.668439] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 5183.098329] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5183.208798] alg: No test for adler32 (adler32-zlib) [ 5184.339814] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 5184.643576] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 5186.335562] Key type lgssc registered [ 5187.547438] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5204.690466] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:58:27 (1787702307) [ 5223.130205] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 5234.827658] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 5280.301575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5280.374242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5281.587700] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5281.643879] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5281.775772] Lustre: lustre-MDT0000: new disk, initializing [ 5281.936226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5281.976557] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5287.438853] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5299.411193] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5299.557924] Lustre: 166487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5299.643645] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5299.648976] Lustre: Skipped 1 previous similar message [ 5299.739699] Lustre: lustre-MDT0001: new disk, initializing [ 5299.835955] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5299.901334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5299.914294] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5305.654267] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5316.780151] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5323.256283] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5331.787367] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5332.237340] Lustre: lustre-OST0000: new disk, initializing [ 5332.246683] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5332.257250] Lustre: 168450:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5332.437959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5338.217704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5338.233465] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5338.386654] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5340.153891] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5351.727551] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5364.196423] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5364.202309] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5364.210271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5364.240191] Lustre: Skipped 1 previous similar message [ 5369.278939] Lustre: server umount lustre-OST0000 complete [ 5374.433115] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5374.460658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5374.478366] Lustre: Skipped 2 previous similar messages [ 5379.521636] Lustre: server umount lustre-MDT0000 complete [ 5383.601493] LustreError: 166479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787702488 with bad export cookie 15661112421614726686 [ 5383.603744] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5383.629763] LustreError: 166479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5384.010902] Lustre: server umount lustre-MDT0001 complete [ 5396.868588] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 5400.071817] Key type lgssc unregistered [ 5400.381957] LNet: 170360:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5400.395681] LNetError: 170360:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5400.422429] LNet: Removed LNI 192.168.204.149@tcp [ 5401.436228] Key type .llcrypt unregistered [ 5401.437830] Key type ._llcrypt unregistered [ 5427.934987] Key type ._llcrypt registered [ 5427.950369] Key type .llcrypt registered [ 5428.091917] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 5429.027389] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5429.138082] alg: No test for adler32 (adler32-zlib) [ 5430.273994] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 5430.520176] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 5432.215264] Key type lgssc registered [ 5433.743925] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5442.121817] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5442.146331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5443.665376] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5448.381583] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5453.794880] LustreError: 171767:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5453.813148] LustreError: 171767:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5458.666868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5458.917678] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5459.216241] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5465.618719] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5474.413880] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5481.774226] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5491.838895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5492.260814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5499.808600] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 5509.001726] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5513.844935] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5520.893173] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5520.935266] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5520.967476] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5520.986456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.017076] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5522.011614] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5522.037027] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5522.060506] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5522.071784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.101283] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5523.177469] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5523.204777] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5523.227141] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5523.233269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.247455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5524.115871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.053971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.018363] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5526.023275] Lustre: Skipped 2 previous similar messages [ 5526.037512] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5526.046798] Lustre: Skipped 2 previous similar messages [ 5526.057078] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5526.067303] Lustre: Skipped 2 previous similar messages [ 5526.078337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.090782] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5526.096386] Lustre: Skipped 2 previous similar messages [ 5527.135404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.255333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.208995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.220740] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5530.225742] Lustre: Skipped 3 previous similar messages [ 5530.250935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5530.257028] Lustre: Skipped 3 previous similar messages [ 5530.259689] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5530.262843] Lustre: Skipped 3 previous similar messages [ 5530.265379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.272179] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5530.275268] Lustre: Skipped 3 previous similar messages [ 5531.888255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.830094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.911082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.027417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.323313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.653195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.723150] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5538.731259] Lustre: Skipped 6 previous similar messages [ 5538.748197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5538.755781] Lustre: Skipped 6 previous similar messages [ 5538.759920] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5538.771968] Lustre: Skipped 6 previous similar messages [ 5538.779192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.796665] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5538.807043] Lustre: Skipped 6 previous similar messages [ 5539.922751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.126352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.274344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.515275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.609743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.637217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.858758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.160688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.584981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.803706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.726228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.736460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.619653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.660692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.603647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5555.609263] Lustre: Skipped 14 previous similar messages [ 5555.623371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5555.631721] Lustre: Skipped 14 previous similar messages [ 5555.635078] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5555.638531] Lustre: Skipped 14 previous similar messages [ 5555.641663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.651263] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5555.663813] Lustre: Skipped 14 previous similar messages [ 5556.633460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.522875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.395983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.533927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.418724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.380738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.278854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.294974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.344594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.519444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.780675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.742427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.628207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.635419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.592803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.645292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.922225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.222337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.634145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.859381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.828526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.772974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.086122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.338878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.592872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.583876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.817877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.818392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.760315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.769648] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5587.787300] Lustre: Skipped 29 previous similar messages [ 5587.808800] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5587.818476] Lustre: Skipped 29 previous similar messages [ 5587.824563] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5587.837211] Lustre: Skipped 29 previous similar messages [ 5587.849342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.865835] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5587.871737] Lustre: Skipped 29 previous similar messages [ 5588.915941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.977625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.120399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.052102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.155996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.236703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.233298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.250528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.354715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.526909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.822327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.968547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.265551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.309696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.522642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.628926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.578437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.726183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.771863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.786286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.673911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.748331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.721734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.762183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.732437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.776979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.670822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.798760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.617853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.674920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.595363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.597321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.459074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.581288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.701558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.800736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.909911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.916303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.101593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.936052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.981492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.060913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.241287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.145109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.575811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.735847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.580878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.513882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.586505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.473962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.278424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.719929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.236957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.855417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.444757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.629281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.909717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.211637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.375037] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5652.397769] Lustre: Skipped 58 previous similar messages [ 5652.417689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5652.430920] Lustre: Skipped 58 previous similar messages [ 5652.444805] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5652.452349] Lustre: Skipped 58 previous similar messages [ 5652.465776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.475651] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5652.491659] Lustre: Skipped 58 previous similar messages [ 5653.808480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.474939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.219046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.986224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.409220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.672949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.622504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.713289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.088172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.193448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.666669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.794454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.830627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.887562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.984881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.823410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.812492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.766042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.787758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.903667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.989125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.011779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.950414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.155029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.376906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.563475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.637031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.554573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.415331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.463477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.381967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.427290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.389976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.503655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.519975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.552149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.593501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.508566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.374470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.485716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.518645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.350595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.236968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.225624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.120905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.063508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.167400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.210391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.088834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.312779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.339677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.433460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.452119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.539073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.453931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.473679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.320460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.302544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.254767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.346139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.308492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.426160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.396891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.405486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.295872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.271874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.447582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.359592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.380844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.364373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.456199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.453988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.438237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.327406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.269428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.158474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.207329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.101274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.961355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.992557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.970932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.944335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.970031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.828087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.775547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.647544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.633754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.616162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.529079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.745995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.781810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.910629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.814814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.842415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.920099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.094935] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5753.114584] LustreError: 171787:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5753.130493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.038127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.142652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.058970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.106779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.162655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.176146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.161691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.245414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.191259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.480366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.326869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.387971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.381826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.643845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.880733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.867270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.911272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.930453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.852172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.867316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.086932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.461347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.551693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.812267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.046408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.128038] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5781.141229] Lustre: Skipped 121 previous similar messages [ 5781.184454] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5781.197757] Lustre: Skipped 121 previous similar messages [ 5781.202066] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5781.212449] Lustre: Skipped 121 previous similar messages [ 5781.217193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.230738] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5781.237038] Lustre: Skipped 121 previous similar messages [ 5782.432319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.458391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.756433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.108866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.265755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.464896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.568959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.113851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.055712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.062864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.001270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.964192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.902406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.968050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.006677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.979894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.003761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.219150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.134254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.342269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.383869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.436966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.601502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.642261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.694404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.686175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.667889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.802477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.809490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.964915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.053198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.009558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.403208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.635603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.648754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.749653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.976593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.262801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.372967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.236694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.353707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.450207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.428921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.772755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.995061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.141662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.443430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.747981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.782355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.765743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.770443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.112242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.064587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.244180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.596383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.639836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.551078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.709872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.996715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.477324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.524706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.606268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.449974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.481405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.518896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.586731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.535281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.656954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.748661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.933414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.826157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.882736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.982045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.035466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.005937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.862512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.826169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.890763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.951936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.885429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.872869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.911635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.094730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.374223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.824357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.982136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.093234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.999628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.106281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.262455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.386811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.363597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.574107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.937563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.333079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.605944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.517428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.824342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.050695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.360779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.377117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.641597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.798251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.289373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.984039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.420356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.792910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.078630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.477071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.892695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.110155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.893415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.173483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.143904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.397538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.536126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.395844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.360053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.224292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.179634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.034656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.030663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.901503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.898990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.825520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.944499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.009941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.141368] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5924.163336] LustreError: 171787:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5924.168151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.020903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.220535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.359643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.473953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.751334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.822374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.813283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.917833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.175448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.143393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.228904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.517149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.640032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.895159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.315843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.698303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.822336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.096469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.001436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.983936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.782041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.844746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.620177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.729335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.795326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.871335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.851530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.784938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.654467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.803819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.898103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.158826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.079479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.075168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.093755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.117975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.207498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.223223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.297673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.248517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.429850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.219033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.187683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.064235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.062246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.018805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.980241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.971330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.005073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.052641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.058145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.030661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.093423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.059570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.179678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.430170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.506526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.592518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.774918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.919567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.990063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.955773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.980026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.015439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.266865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.515726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.609834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.623104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.649194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.719374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.707074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.700692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.574393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.471176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.616854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.901171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.991076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.026452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.212142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.333587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.349158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.407175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.391263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.314203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.300940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.345830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.249528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.297442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.505171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.496122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.660088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.865452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.035146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.018623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.078939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.138416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.977053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.936189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.999702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.025824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.167505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.475311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.663078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.805756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.874922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.137446] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6037.145234] Lustre: Skipped 233 previous similar messages [ 6037.168986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.156618] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6038.177692] Lustre: Skipped 234 previous similar messages [ 6038.190508] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6038.198590] Lustre: Skipped 234 previous similar messages [ 6038.210593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.223849] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6038.233931] Lustre: Skipped 234 previous similar messages [ 6039.298883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.187324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.356058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.208965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.222232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.269221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.240107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.413975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.352845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.357042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.382081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.560610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.781451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.907182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.859067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.982353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.182725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.327719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.422772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.568924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.821358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.929675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.205638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.202096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.511090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.620614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.985092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.140315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.412222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.362906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.287811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.070045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.114037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.369348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.611309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.846577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.980187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.231940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.282477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.370631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.597863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.736708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.738233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.753146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.750708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.601331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.418643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.385270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.162930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.059342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.040813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.218648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.443808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.684899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.894872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.984433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.952768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.882610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.757854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.819435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.792431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.859248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.882360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.865012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.077369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.973986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.158947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.325072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.354597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.515909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.495068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.457436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.475462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.435344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.598964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.610186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.736615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.813448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.661341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.521707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.627815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.648820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.692429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.473496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.398293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.325180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.198426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.994317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.913053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.851932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.899397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.817885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.819301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.763309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.931803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.924491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.834642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.830192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.591481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.403984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.173590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.079190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.037676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.008247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.034094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.982176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.837764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.704605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.572099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.513143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.487900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.618441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.744325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.883556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.825873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.850904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.915498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.880892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.869908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.878510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.972852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.940769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.969820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.041140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.036331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.954497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.980569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.087661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.191414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.141921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.234258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.212700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.268656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.281187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.330342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.516023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.553726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.642520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.775744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.874441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.916262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.851498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.762102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.737027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.789874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.869975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.836685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.939248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.940118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.063930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.372751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.565657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.425862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.388296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.434481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.680426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.827546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.875703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.686764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.605235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.743747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.307114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.992209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.348384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.918579] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 6210.945407] LustreError: 171787:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 6210.947064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.299888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.724642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.995649] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6215.011659] LustreError: 171787:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6215.018882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.959531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.127713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.350808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.390381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.440595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.398981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.494046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.308906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.398666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.336267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.530085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.470571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.586901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.514556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.488591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.331790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.380216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.325643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.283629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.370079] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6235.377992] LustreError: 171787:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6235.390781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.612380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.588464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.511888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.404920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.477134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.411528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.307508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.271553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.228139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.164150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.214839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.106215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.127251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.142632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.137237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.166868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.368436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.392699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.510348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.437739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.560758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.643689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.840981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.128861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.155975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.189825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.092354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.225050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.149788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.129301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.139149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.220178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.343922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.456778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.452757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.516692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.533774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.656723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.737336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.845280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.816971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.965360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.947811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.034986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.266128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.099804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.042943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.986810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.856914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.862047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.975127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.032638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.047345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.008827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.974773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.752444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.735630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.818264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.804326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.723218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.778561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.830430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.853967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.985890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.036981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.025528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.113062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.119556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.342669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.342250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.386855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.446591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.460145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.361554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.283405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.168893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.199051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.251240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.297400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.392884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.594968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.683926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.712163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.762454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.951735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.998119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.111400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.208335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.211876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.454751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.649668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.663438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.750234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.661791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.676749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.595747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.689991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.796879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.677346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.612645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.478481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.569729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.491965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.451293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.619563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.591095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.420062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.205608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.198045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.227218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.254605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.162271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.227230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.109822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.014455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.956783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.936436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.763392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.749711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.770572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.730089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.757157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.798969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.858841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.181528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.233216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.097765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.140617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.213692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.235811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.176845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.128741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.120280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.067363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.361389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.651368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.814041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.974699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.905977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.913065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.731226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.572540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.461869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.507645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.441862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.454090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.382504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.358578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.316794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.351558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.644045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.574525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.790769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.017248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.187392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.968938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.816288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.877534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.804780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.902806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.901736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.970216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.019609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.934485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.073836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.059349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.049516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.089760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.179244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.307064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.139403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.240213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.247386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.285301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.192101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.036944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.865115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.102801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.209422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.228849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.383054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.493133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.475264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.345507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.463561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.445344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.462491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.274987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.266872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.183273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.276566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.204153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.076257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.137154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.166789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.019655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.840932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.712262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.724677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.852857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.069672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.110880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.161781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.245731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.140791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.139945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.945278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.870343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.799740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.787504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.703424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.699504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.498599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.555385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.436682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.402778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.365547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.362433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.420311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.524574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.461518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.348331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.254610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.213077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.276158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.314079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.337400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.269844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.405994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.452192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.286307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.309468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.071906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.079416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.147548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.172282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.221696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.212807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.192264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.116157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.243177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.354698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.337490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.394158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.699716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.734884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.742096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.042802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.080583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.109130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.245796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.425694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.542371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.669180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.801703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.578272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.523496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.437170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.530048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.464680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.554534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.340894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.403280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.513772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.581078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.612863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.502687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.408610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.327364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.282873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.410887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.516392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.351421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.468849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.441323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.357746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.543405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.630972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.686931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.644293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.703472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.665846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.699923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.739347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.829203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.909694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.973368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.823559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.946717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.754884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.786406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.848997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.787707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.681447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.559252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.565627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.434649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.408850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.561065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.753396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.715578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.672967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.659855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.581042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.650430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.721075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.829032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.761950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.826039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.917277] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6549.922262] Lustre: Skipped 498 previous similar messages [ 6549.962378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.944797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6550.953848] Lustre: Skipped 498 previous similar messages [ 6550.960899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6550.966437] Lustre: Skipped 498 previous similar messages [ 6550.970848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.981555] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6550.990587] Lustre: Skipped 498 previous similar messages [ 6552.213320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.351049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.371278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.494612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.513300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.350260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.331738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.254639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.212140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.219845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.129802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.200820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.176867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.219604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.299887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.384101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.420112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.680187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.788780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.739509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.758163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.841113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.963775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.244613] LustreError: 174252:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7068.259935] LustreError: 171787:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7809.060176] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7809.064249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7809.073554] LustreError: Skipped 1 previous similar message [ 7809.097297] Lustre: Skipped 23 previous similar messages [ 7809.125183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7812.580968] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7812.594206] LustreError: Skipped 6 previous similar messages [ 7812.603636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7818.212395] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7818.234116] Lustre: Skipped 3 previous similar messages [ 7823.327298] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7823.334681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7823.359081] Lustre: Skipped 1 previous similar message [ 7823.600870] Lustre: server umount lustre-OST0000 complete [ 7829.985239] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7829.996592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7834.963198] Lustre: server umount lustre-MDT0000 complete [ 7838.690209] LustreError: 221801:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7838.705774] LustreError: 221801:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7839.121216] LustreError: 171749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787704943 with bad export cookie 10352579815001426917 [ 7839.143784] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7839.149901] LustreError: 171749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7839.626809] Lustre: server umount lustre-MDT0001 complete [ 7853.781068] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 7856.965528] Key type lgssc unregistered [ 7857.289882] LNet: 226739:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7857.300852] LNetError: 226739:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7858.348721] LNet: Removed LNI 192.168.204.149@tcp [ 7859.570761] Key type .llcrypt unregistered [ 7859.575721] Key type ._llcrypt unregistered [ 7889.832457] Key type ._llcrypt registered [ 7889.835032] Key type .llcrypt registered [ 7889.976590] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 7890.926221] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7890.981840] alg: No test for adler32 (adler32-zlib) [ 7892.050631] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 7892.270585] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 7893.967217] Key type lgssc registered [ 7895.531983] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7904.197473] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7904.233350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7905.953522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7911.124779] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 7915.492655] LustreError: 228150:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7915.524395] LustreError: 228150:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7920.608156] LustreError: 228149:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7920.926138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7921.451820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7926.458378] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 7935.975564] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7942.947806] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7951.068959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7951.276851] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 7951.392168] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7956.508489] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7958.466365] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 7965.594791] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7970.772235] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7973.419791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7973.436135] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7973.446128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7973.457844] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7973.463487] LustreError: 228169:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7976.940485] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7976.945330] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7976.957322] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7976.992123] Lustre: Skipped 1 previous similar message [ 7982.056107] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7982.073220] Lustre: Skipped 3 previous similar messages [ 7982.299986] Lustre: server umount lustre-OST0000 complete [ 7993.567203] LustreError: 228169:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8017.644077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8018.186839] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8024.924878] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8031.467261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8031.485609] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8031.488363] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8031.515531] Lustre: Skipped 1 previous similar message [ 8033.638347] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8046.245457] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 9 sec [ 8054.135958] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c8ccea57000.ost_server_uuid 50 [ 8056.106656] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c8ccea57000.ost_server_uuid in FULL state after 0 sec [ 8061.411931] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8061.422493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8061.431946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8062.432757] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8063.318962] Lustre: server umount lustre-OST0000 complete [ 8067.553435] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8067.556793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8067.600183] Lustre: Skipped 2 previous similar messages [ 8067.622187] Lustre: Skipped 3 previous similar messages [ 8073.295126] Lustre: server umount lustre-MDT0000 complete [ 8077.795664] LustreError: 229588:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8077.838890] LustreError: 229588:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8078.816909] LustreError: 232608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705183 with bad export cookie 3374022738959621678 [ 8078.825857] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8078.828947] LustreError: 232608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8079.320405] Lustre: server umount lustre-MDT0001 complete [ 8089.709179] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 8092.963979] Key type lgssc unregistered [ 8093.331758] LNet: 233359:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8093.338160] LNetError: 233359:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8093.359161] LNet: Removed LNI 192.168.204.149@tcp [ 8094.533864] Key type .llcrypt unregistered [ 8094.537963] Key type ._llcrypt unregistered [ 8131.008931] Key type ._llcrypt registered [ 8131.012535] Key type .llcrypt registered [ 8131.142584] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 8132.744123] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8132.791243] alg: No test for adler32 (adler32-zlib) [ 8134.109796] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 8134.403606] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 8136.159185] Key type lgssc registered [ 8137.479035] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8146.318101] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8146.352741] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8147.876101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8152.143386] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8158.181864] LustreError: 235098:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8158.195728] LustreError: 235098:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8161.771028] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8162.068164] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8166.348491] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8169.227474] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8175.746540] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8176.124959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8181.862431] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8191.194868] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8191.402308] Lustre: lustre-OST0001: new disk, initializing [ 8191.412419] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8191.419962] Lustre: 237401:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8191.556203] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8192.580088] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8198.784617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8198.792613] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8198.959293] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8199.333852] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8211.551386] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8222.272873] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8239.073950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8239.089197] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8239.119987] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8240.106865] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8240.122449] Lustre: Skipped 2 previous similar messages [ 8240.125475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8240.137758] Lustre: Skipped 3 previous similar messages [ 8245.219118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8245.238300] Lustre: Skipped 2 previous similar messages [ 8250.340911] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8250.357537] Lustre: Skipped 3 previous similar messages [ 8251.359489] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8251.602994] Lustre: server umount lustre-MDT0000 complete [ 8255.460172] LustreError: 235098:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8255.489278] LustreError: 235098:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8255.832679] LustreError: 235080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705360 with bad export cookie 15676369488412159632 [ 8255.835104] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8255.858674] LustreError: 235080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8256.443057] Lustre: server umount lustre-MDT0001 complete [ 8270.431820] Lustre: server umount lustre-OST0000 complete [ 8285.310473] Lustre: server umount lustre-OST0001 complete [ 8310.546387] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 8322.811583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8323.354768] LustreError: 240490:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8323.450927] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8328.256433] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8328.672962] LustreError: 240491:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8333.802051] LustreError: 240490:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8338.504381] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8338.941632] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8343.878637] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8347.649819] Lustre: 241629:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8355.203276] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8355.589827] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8361.909067] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8363.770632] LustreError: 241982:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8368.902270] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8371.450838] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8378.327526] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8387.777980] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8391.949779] Lustre: 243492:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8405.472647] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8405.480806] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8405.500680] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8407.530354] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8407.555138] Lustre: Skipped 2 previous similar messages [ 8409.005021] Lustre: server umount mylustre-MDT0000 complete [ 8412.649041] LustreError: 240490:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8412.676451] LustreError: 240490:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8413.951756] LustreError: 240467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705518 with bad export cookie 15676369488412162439 [ 8413.973736] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8413.979040] LustreError: 240467:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8414.539556] Lustre: server umount mylustre-MDT0001 complete [ 8429.770118] Lustre: server umount mylustre-OST0000 complete [ 8433.640193] Lustre: 234363:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787705522/real 1787705522] req@ffff9b929db9d180 x1874544808849792/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787705538 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8433.713603] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8434.813597] Lustre: server umount mylustre-OST0001 complete [ 8461.371484] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 8474.956774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8475.665246] LustreError: 245791:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8475.761840] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8475.767777] Lustre: Skipped 1 previous similar message [ 8480.753236] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8490.717666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8490.989153] Lustre: tfs-MDT0001: Not available for connect from 0@lo (not set up) [ 8491.001638] Lustre: Skipped 4 previous similar messages [ 8496.856031] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8500.438705] Lustre: 246932:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8508.207734] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8515.977620] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8520.886739] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8525.215149] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8533.494658] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8542.028766] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8546.132997] Lustre: 248798:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8555.952157] Lustre: server umount tfs-MDT0000 complete [ 8556.519158] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8556.552908] LustreError: 245788:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8556.567921] LustreError: 245788:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 8559.802508] LustreError: 245767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705664 with bad export cookie 15676369488412164105 [ 8559.814114] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8559.822634] LustreError: 245767:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8560.095723] Lustre: server umount tfs-MDT0001 complete [ 8573.513908] Lustre: server umount tfs-OST0000 complete [ 8587.952723] Lustre: server umount tfs-OST0001 complete [ 8611.712334] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 8623.732935] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8624.446686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8624.459249] Lustre: Skipped 3 previous similar messages [ 8629.746346] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8638.888560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8643.724621] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8646.755423] Lustre: 252176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8654.457115] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8661.793050] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8669.370988] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8672.509313] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8680.821275] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8689.161958] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8693.421412] Lustre: 254039:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8700.934713] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 20:56:43 (1787705803) [ 8718.832135] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8718.843772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8718.856169] Lustre: Skipped 6 previous similar messages [ 8718.864086] Lustre: Skipped 2 previous similar messages [ 8723.854374] Lustre: server umount lustre-MDT0000 complete [ 8723.939333] LustreError: 253554:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8723.966060] LustreError: 253554:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8728.395699] LustreError: 251015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787705833 with bad export cookie 15676369488412165547 [ 8728.409573] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8728.412246] LustreError: 251015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8735.082806] Lustre: server umount lustre-MDT0001 complete [ 8745.847860] Lustre: server umount lustre-OST0000 complete [ 8756.687507] Lustre: server umount lustre-OST0001 complete [ 8763.538151] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 8775.560544] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 8815.485707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8815.895687] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8815.931111] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8816.021818] Lustre: lustre-MDT0000: new disk, initializing [ 8816.124049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8816.128116] Lustre: Skipped 3 previous similar messages [ 8816.139677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8820.706874] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8832.646808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8832.771618] Lustre: 257209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8832.851273] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8832.859360] Lustre: Skipped 1 previous similar message [ 8832.966520] Lustre: lustre-MDT0001: new disk, initializing [ 8833.162327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8833.169323] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8837.631201] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8847.701274] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8847.958873] Lustre: lustre-OST0000: new disk, initializing [ 8847.962883] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8847.968694] Lustre: 258788:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8849.550438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8849.559640] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8849.677284] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8854.371266] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8866.428801] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8866.673722] Lustre: lustre-OST0001: new disk, initializing [ 8866.677966] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8866.685135] Lustre: 259659:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8868.043890] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8868.050918] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8868.215306] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8874.939506] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 8889.878557] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:59:52 (1787705992) [ 8907.385967] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:00:10 (1787706010) [ 8924.649599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8924.652617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8924.665856] Lustre: Skipped 3 previous similar messages [ 8924.677121] Lustre: Skipped 5 previous similar messages [ 8929.233083] Lustre: server umount lustre-OST0000 complete [ 8940.746191] Lustre: server umount lustre-MDT0000 complete [ 8945.309334] LustreError: 257203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787706050 with bad export cookie 15676369488412166870 [ 8945.319326] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8945.327985] LustreError: 257203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8952.062192] Lustre: server umount lustre-MDT0001 complete [ 8965.040206] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 9042.402388] Lustre: server umount lustre-OST0001 complete [ 9050.201741] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 9066.727786] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 9106.731536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9106.969810] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9107.003495] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9107.081626] Lustre: lustre-MDT0000: new disk, initializing [ 9107.163070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9107.166743] Lustre: Skipped 3 previous similar messages [ 9107.177391] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9111.898216] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9123.120708] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9123.286393] Lustre: 264497:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9123.358748] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9123.368711] Lustre: Skipped 1 previous similar message [ 9123.481382] Lustre: lustre-MDT0001: new disk, initializing [ 9123.681408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9123.695754] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9128.797631] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9138.809305] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9145.809273] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9153.454281] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9153.742686] Lustre: lustre-OST0000: new disk, initializing [ 9153.746612] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9153.752427] Lustre: 266460:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9155.638065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9155.646297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9155.812823] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9160.435368] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9170.028565] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9181.668096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9181.669631] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9181.674302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9181.674310] Lustre: Skipped 7 previous similar messages [ 9181.718727] Lustre: Skipped 5 previous similar messages [ 9183.364506] Lustre: server umount lustre-OST0000 complete [ 9190.380037] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9190.393839] LustreError: Skipped 1 previous similar message [ 9193.670767] Lustre: server umount lustre-MDT0000 complete [ 9197.027660] LustreError: 264503:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9197.062631] LustreError: 264503:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 9197.840754] LustreError: 264489:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787706302 with bad export cookie 15676369488412168347 [ 9197.856432] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9198.210358] Lustre: server umount lustre-MDT0001 complete [ 9210.133720] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 9212.915304] Key type lgssc unregistered [ 9213.246905] LNet: 268213:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9213.254956] LNetError: 268213:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9213.277435] LNet: Removed LNI 192.168.204.149@tcp [ 9214.167952] Key type .llcrypt unregistered [ 9214.170423] Key type ._llcrypt unregistered [ 9242.025529] Key type ._llcrypt registered [ 9242.030304] Key type .llcrypt registered [ 9242.149758] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 9243.253395] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9243.329165] alg: No test for adler32 (adler32-zlib) [ 9244.414390] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 9244.669459] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 9246.399229] Key type lgssc registered [ 9247.597749] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9255.632859] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9255.659054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9257.163178] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9261.809234] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9266.659523] LustreError: 269670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9266.696771] LustreError: 269670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9271.327448] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9271.718151] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9277.205454] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9286.301048] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9293.243595] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9300.678549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9301.010482] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9302.003165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9307.712924] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9316.116326] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9317.353641] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9321.446421] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9321.458587] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9321.474418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9326.240985] Lustre: server umount lustre-OST0000 complete [ 9331.684538] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9331.704298] Lustre: Skipped 1 previous similar message [ 9331.713323] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9331.721059] Lustre: Skipped 2 previous similar messages [ 9336.806462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9336.820265] Lustre: Skipped 1 previous similar message [ 9337.213506] Lustre: server umount lustre-MDT0000 complete [ 9341.391480] LustreError: 272189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787706446 with bad export cookie 11059443227807551699 [ 9341.393279] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9341.407880] LustreError: 272189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9341.776152] Lustre: server umount lustre-MDT0001 complete [ 9352.670105] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 9356.042339] Key type lgssc unregistered [ 9356.372474] LNet: 273244:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9356.381203] LNetError: 273244:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9356.400605] LNet: Removed LNI 192.168.204.149@tcp [ 9357.469066] Key type .llcrypt unregistered [ 9357.478516] Key type ._llcrypt unregistered [ 9376.134719] Key type ._llcrypt registered [ 9376.137622] Key type .llcrypt registered [ 9376.273858] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 9377.531026] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9377.597105] alg: No test for adler32 (adler32-zlib) [ 9378.778162] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 9379.037236] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 9380.687402] Key type lgssc registered [ 9382.357897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9392.669773] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9392.684969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9408.223484] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9413.344429] LustreError: 274435:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.149@tcp: failed processing log, type 4: rc = -110 [ 9440.031876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9449.692467] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9458.683977] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9461.178578] Lustre: server umount lustre-OST0000 complete [ 9480.784098] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [ 9484.380954] Key type lgssc unregistered [ 9484.742641] LNet: 276057:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9484.748966] LNetError: 276057:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9484.768499] LNet: Removed LNI 192.168.204.149@tcp [ 9485.642314] Key type .llcrypt unregistered [ 9485.647557] Key type ._llcrypt unregistered [ 9529.335669] Key type ._llcrypt registered [ 9529.339320] Key type .llcrypt registered [ 9529.462699] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 9530.616172] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9530.633950] alg: No test for adler32 (adler32-zlib) [ 9531.672968] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [ 9531.930890] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [ 9533.631175] Key type lgssc registered [ 9535.147595] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9548.786303] loop0: detected capacity change from 0 to 204800000 [ 9548.975900] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9548.993810] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9549.407793] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9550.386343] loop0: detected capacity change from 0 to 204800000 [ 9550.585477] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9550.613854] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9551.174516] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9552.181925] loop0: detected capacity change from 0 to 204800000 [ 9552.371411] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9552.390834] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9552.515882] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9553.622771] loop0: detected capacity change from 0 to 204800000 [ 9553.817813] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9553.985487] print_req_error: 1 callbacks suppressed [ 9553.985498] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9555.149308] loop0: detected capacity change from 0 to 204800000 [ 9555.205481] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9560.291774] loop0: detected capacity change from 0 to 204800000 [ 9560.347483] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9566.103688] loop0: detected capacity change from 0 to 204800000 [ 9566.139945] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9571.196066] loop0: detected capacity change from 0 to 204800000 [ 9571.222387] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9575.760158] loop0: detected capacity change from 0 to 204800000 [ 9575.840919] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9575.879495] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9575.935156] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9575.944994] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9575.954198] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9575.964297] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9575.977443] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9575.988112] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9575.998909] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9576.007961] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9576.019617] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9576.034926] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9576.056712] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9576.071380] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9576.081417] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9576.091178] Lustre: 279461:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9583.180782] Lustre: server umount MGS complete [ 9584.258260] loop0: detected capacity change from 0 to 204800000 [ 9584.355543] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9585.066755] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9585.141935] Lustre: lustre-MDT0000: Aborting client recovery [ 9586.314100] loop1: detected capacity change from 0 to 204800000 [ 9586.561069] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9586.643041] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9586.651200] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9586.662992] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9586.676052] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9586.690831] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9586.724194] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9586.744428] Lustre: 279893:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9586.765803] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9586.901475] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9586.947917] Lustre: lustre-MDT0001: Aborting client recovery [ 9587.898048] loop2: detected capacity change from 0 to 204800000 [ 9588.051171] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9588.140928] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9588.173662] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9588.198554] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9588.223508] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9588.236550] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9588.251564] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9588.270988] Lustre: 279972:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9588.431255] Lustre: Found index 0 for lustre-OST0000, updating log [ 9588.513942] Lustre: 279990:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9588.620416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9588.640186] Lustre: lustre-OST0000: Aborting recovery [ 9589.746573] loop3: detected capacity change from 0 to 204800000 [ 9589.907474] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9590.061872] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9590.087749] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9590.106907] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9590.119428] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9590.134490] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9590.148124] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9590.168117] Lustre: 280053:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9590.213347] Lustre: Found index 1 for lustre-OST0001, updating log [ 9590.269807] Lustre: 280059:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9590.405681] Lustre: lustre-OST0001: Aborting recovery [ 9594.473141] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9594.554484] LustreError: 279837:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9594.577611] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9594.622570] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9594.707432] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9596.545972] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9596.546848] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9599.077235] LustreError: 279804:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 592us: rc = -2 [ 9599.589680] LustreError: 279800:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 675us: rc = -2 [ 9599.606188] LustreError: 279800:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 9612.258399] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9612.263625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9612.273076] Lustre: Skipped 3 previous similar messages [ 9612.283080] Lustre: Skipped 3 previous similar messages [ 9617.376060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9617.381453] Lustre: Skipped 3 previous similar messages [ 9618.613385] Lustre: server umount lustre-MDT0000 complete [ 9619.760301] LustreError: 279786:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787706724 with bad export cookie 3898740649394605223 [ 9619.783414] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9622.497522] LustreError: 279802:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9622.499080] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9622.534548] LustreError: 279802:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9622.555915] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9622.564285] Lustre: Skipped 1 previous similar message [ 9624.551731] LustreError: 279800:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9624.581512] LustreError: 279800:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9629.670959] LustreError: 279800:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9629.675205] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9629.694983] Lustre: Skipped 3 previous similar messages [ 9634.272322] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9634.640839] Lustre: server umount lustre-MDT0001 complete [ 9650.143221] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9650.278694] Lustre: server umount lustre-OST0000 complete [ 9657.505619] Lustre: server umount lustre-OST0001 complete [ 9666.556446] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:12:49 (1787706769) [ 9702.776857] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [ 9716.142781] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [ 9764.139415] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9764.633553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9764.725580] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9764.914433] Lustre: lustre-MDT0000: new disk, initializing [ 9765.081094] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9765.095635] Lustre: Skipped 1 previous similar message [ 9765.152675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9771.459530] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9786.750054] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9786.928762] Lustre: 284105:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9786.969684] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9786.974305] Lustre: Skipped 1 previous similar message [ 9787.042316] Lustre: lustre-MDT0001: new disk, initializing [ 9787.144354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9787.200708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9787.218104] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9791.870911] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9801.797712] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9807.688577] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9814.912896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9815.196927] Lustre: lustre-OST0000: new disk, initializing [ 9815.201297] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9815.212631] Lustre: 286069:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9815.310091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9821.488567] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9821.772788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9821.801720] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9821.900697] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9832.429029] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9838.029532] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9848.336931] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9852.587426] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9863.200704] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9904.097060] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9904.109462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9904.116433] Lustre: Skipped 2 previous similar messages [ 9904.143080] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9907.693357] Lustre: server umount lustre-OST0000 complete [ 9914.341921] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9914.349154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9914.371400] Lustre: Skipped 1 previous similar message [ 9914.387898] Lustre: Skipped 2 previous similar messages [ 9918.925537] Lustre: server umount lustre-MDT0000 complete [ 9919.457416] LustreError: 284115:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9919.488984] LustreError: 284115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 9923.312106] LustreError: 284898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787707028 with bad export cookie 3898740649394607946 [ 9923.318737] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9923.327863] LustreError: 284898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 9923.772608] Lustre: server umount lustre-MDT0001 complete [ 9935.751626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9939.807739] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9944.772066] Lustre: server umount MGS complete [ 9958.349767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9958.902839] LustreError: 289824:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9958.922758] LustreError: 289837:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9959.001819] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9963.956265] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9969.128880] LustreError: 289825:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9969.167838] LustreError: 289825:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9973.150117] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9978.432073] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [ 9987.005182] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9993.826881] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10001.563876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10001.969109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10001.977562] Lustre: Skipped 1 previous similar message [10008.199641] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10015.969327] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10048.999846] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10049.015569] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10049.017264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10049.018948] Lustre: Skipped 1 previous similar message [10053.965302] Lustre: server umount lustre-OST0000 complete [10059.236584] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10059.245029] Lustre: Skipped 1 previous similar message [10064.555296] Lustre: server umount lustre-MDT0000 complete [10068.453618] LustreError: 292364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10068.473797] LustreError: 292364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10068.566663] LustreError: 289804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787707173 with bad export cookie 3898740649394609899 [10068.579470] LustreError: 289804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10068.579856] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10069.006178] Lustre: server umount lustre-MDT0001 complete [10080.640836] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [10083.415684] Key type lgssc unregistered [10083.674498] LNet: 293756:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10083.689876] LNetError: 293756:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10083.712552] LNet: Removed LNI 192.168.204.149@tcp [10084.378344] Key type .llcrypt unregistered [10084.380583] Key type ._llcrypt unregistered [10130.876661] Key type ._llcrypt registered [10130.886425] Key type .llcrypt registered [10131.024299] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [10150.775996] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [10151.811639] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10151.905111] alg: No test for adler32 (adler32-zlib) [10153.000257] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [10153.231981] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [10154.879193] Key type lgssc registered [10156.295123] Lustre: Echo OBD driver; http://www.lustre.org/ [10201.387990] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10201.418824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10202.623406] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10202.675981] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10202.812601] Lustre: lustre-MDT0000: new disk, initializing [10202.923629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10202.941319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10207.486745] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10219.651911] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10219.819415] Lustre: 297704:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10219.913226] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [10219.923265] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10219.939288] Lustre: Skipped 1 previous similar message [10220.057401] Lustre: lustre-MDT0001: new disk, initializing [10220.195045] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10220.237245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10220.252851] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10226.047742] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10238.631118] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10247.369539] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10255.665285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10256.125610] Lustre: lustre-OST0000: new disk, initializing [10256.135879] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10256.161658] Lustre: 299669:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10256.341612] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10263.880948] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10265.191989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10265.221733] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10265.389514] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10274.420685] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10281.851764] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10292.171431] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10296.910992] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10303.155337] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10357.731839] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10357.747721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10357.785529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10357.799275] Lustre: Skipped 1 previous similar message [10359.737462] Lustre: server umount lustre-OST0000 complete [10367.968861] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10367.976077] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10367.987265] Lustre: Skipped 2 previous similar messages [10368.001286] Lustre: Skipped 1 previous similar message [10373.060875] Lustre: server umount lustre-MDT0000 complete [10378.035380] LustreError: 297695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787707482 with bad export cookie 13875027287161315065 [10378.049495] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10378.058068] LustreError: 297695:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10378.703446] Lustre: server umount lustre-MDT0001 complete [10390.764186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10395.743656] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10402.086758] Lustre: server umount MGS complete [10414.431656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10414.981020] LustreError: 303430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10415.014067] LustreError: 303430:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10415.047725] LustreError: 303443:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10415.142145] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10419.904874] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10420.193657] LustreError: 303431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10420.213635] LustreError: 303431:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10425.311878] LustreError: 303430:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10430.439923] LustreError: 303431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10430.551913] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10431.105768] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10436.213514] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10444.668489] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10452.189566] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10459.985224] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10460.383585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10468.188942] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10476.851620] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10506.724181] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10506.728885] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10506.735165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10506.745128] LustreError: Skipped 1 previous similar message [10511.624140] Lustre: server umount lustre-OST0000 complete [10516.972181] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10516.977541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10516.990598] Lustre: Skipped 2 previous similar messages [10517.000316] Lustre: Skipped 1 previous similar message [10527.202622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10527.206519] Lustre: Skipped 4 previous similar messages [10530.271928] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10530.536736] Lustre: server umount lustre-MDT0000 complete [10532.322303] LustreError: 304162:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10532.339481] LustreError: 304162:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10534.895251] LustreError: 303411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787707639 with bad export cookie 13875027287161316913 [10534.907167] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10534.913559] LustreError: 303411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10535.231272] Lustre: server umount lustre-MDT0001 complete [10548.738926] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [10552.028637] Key type lgssc unregistered [10552.504807] LNet: 307364:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10552.518622] LNetError: 307364:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10552.548295] LNet: Removed LNI 192.168.204.149@tcp [10553.917901] Key type .llcrypt unregistered [10553.919345] Key type ._llcrypt unregistered [10578.634384] Key type ._llcrypt registered [10578.635853] Key type .llcrypt registered [10578.771550] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [10595.434144] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [10596.458738] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10596.513000] alg: No test for adler32 (adler32-zlib) [10597.548721] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [10597.851475] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [10599.535306] Key type lgssc registered [10600.875170] Lustre: Echo OBD driver; http://www.lustre.org/ [10630.671973] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [10648.441853] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10648.470755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10649.798725] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10649.824804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10649.921224] Lustre: lustre-MDT0000: new disk, initializing [10650.035991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10650.076596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10655.017106] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [10666.726651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10667.029677] Lustre: lustre-OST0000: new disk, initializing [10667.034415] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10667.040268] Lustre: Skipped 1 previous similar message [10667.045491] Lustre: 311402:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10667.125392] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10674.268831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10674.280873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10674.320414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10675.031318] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11657.188024] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11657.209268] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11659.544860] Lustre: server umount lustre-MDT0000 complete [11663.696499] LustreError: 312262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787708768 with bad export cookie 13856669565914692861 [11663.705766] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11663.711825] LustreError: 312262:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11663.886468] Lustre: server umount lustre-OST0000 complete [11691.853482] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [11702.172700] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [11746.231702] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11746.538720] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11746.572188] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11746.684264] Lustre: lustre-MDT0000: new disk, initializing [11746.830551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11746.856747] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11752.299563] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11764.809678] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11764.965167] Lustre: 315611:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11765.067782] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11765.074586] Lustre: Skipped 1 previous similar message [11765.226285] Lustre: lustre-MDT0001: new disk, initializing [11765.339913] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11765.380102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11765.397567] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11770.514416] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11781.527443] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11789.303147] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11797.360041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11797.831047] Lustre: lustre-OST0000: new disk, initializing [11797.836444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11797.847282] Lustre: 317571:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11797.983172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11803.228961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11803.250275] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11803.413641] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11804.593550] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11814.799625] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11819.798989] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11820.066884] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11825.361520] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11825.707454] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11829.221630] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11829.227091] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11829.244785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11829.283941] Lustre: Skipped 1 previous similar message [11834.341675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11834.347243] Lustre: Skipped 1 previous similar message [11839.461629] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11839.478843] Lustre: Skipped 2 previous similar messages [11842.015149] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11842.260957] Lustre: server umount lustre-OST0000 complete [11847.137739] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11847.148041] LustreError: Skipped 1 previous similar message [11847.155626] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11847.185456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11849.698151] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11853.196688] Lustre: server umount lustre-MDT0000 complete [11857.322156] LustreError: 317569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787708962 with bad export cookie 13856669565917633687 [11857.327383] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11857.333853] LustreError: 317569:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11857.762700] Lustre: server umount lustre-MDT0001 complete [11861.269212] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11863.437519] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:49:26 (1787708966) [11871.186192] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11871.644418] LustreError: 319875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11871.664097] LustreError: 319875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11871.722816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11876.638807] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11876.833429] LustreError: 319876:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11881.962397] LustreError: 319875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11886.183486] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11886.688148] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11891.848191] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11900.854755] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11908.892154] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11920.004932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11920.582925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11929.908062] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11941.705435] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11952.822972] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11953.086192] Lustre: lustre-OST0001: new disk, initializing [11953.092890] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11953.101346] Lustre: 322701:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11953.245165] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11960.548413] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11960.558854] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11960.646959] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11960.674626] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11961.705375] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11961.708569] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [11962.790624] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11962.803700] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11964.839722] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11964.852851] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11969.064203] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11969.084724] LustreError: 323000:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11973.467064] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11973.676375] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11977.261079] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11977.267783] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11981.623763] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c8ccea50800.ost_server_uuid 50 [11984.102882] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c8ccea50800.ost_server_uuid in FULL state after 0 sec [11991.214563] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c8ccea50800.ost_server_uuid 50 [11993.626906] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c8ccea50800.ost_server_uuid in IDLE state after 0 sec [11993.700285] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11993.739197] LustreError: 323006:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [11995.876965] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12009.241358] Lustre: lustre-OST0001: Not available for connect from 192.168.204.49@tcp (stopping) [12009.250633] Lustre: Skipped 2 previous similar messages [12010.471667] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12010.487494] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12015.364975] Lustre: server umount lustre-OST0001 complete [12017.130206] LustreError: 322708:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12017.152669] LustreError: 322708:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12022.254332] LustreError: 321692:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12022.256623] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12022.296597] LustreError: 321692:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12022.329595] Lustre: Skipped 2 previous similar messages [12027.361649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12027.373823] Lustre: Skipped 7 previous similar messages [12028.184203] Lustre: server umount lustre-OST0000 complete [12035.044422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12035.055091] LustreError: Skipped 1 previous similar message [12038.972816] Lustre: server umount lustre-MDT0000 complete [12042.721203] LustreError: 319875:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12042.735974] LustreError: 319875:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12042.836989] LustreError: 319858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787709147 with bad export cookie 13856669565917634569 [12042.846605] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12042.850078] LustreError: 319858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12043.267493] Lustre: server umount lustre-MDT0001 complete [12053.278391] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [12056.309165] Key type lgssc unregistered [12056.660793] LNet: 325379:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12056.668737] LNetError: 325379:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12056.694773] LNet: Removed LNI 192.168.204.149@tcp [12057.692205] Key type .llcrypt unregistered [12057.700303] Key type ._llcrypt unregistered [12083.989601] Key type ._llcrypt registered [12083.995191] Key type .llcrypt registered [12084.076177] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [12085.223099] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12085.294733] alg: No test for adler32 (adler32-zlib) [12086.389458] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [12086.789168] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [12088.463172] Key type lgssc registered [12089.470326] Lustre: Echo OBD driver; http://www.lustre.org/ [12097.557441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12097.573111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12099.034245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12103.292904] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12109.297442] LustreError: 326795:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12109.324154] LustreError: 326795:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12111.338752] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12114.403614] LustreError: 326794:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12118.541993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12118.944453] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12124.628941] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12134.745811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12135.180975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12142.312773] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12143.411548] LustreError: 328468:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12144.459098] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12149.742673] LustreError: 328468:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12149.768713] LustreError: 328468:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12150.638720] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12154.870684] LustreError: 328468:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12154.905587] LustreError: 328468:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12158.702488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12159.072319] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12164.108855] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12165.812843] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12174.633856] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12182.572492] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12182.826325] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12302.319211] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12302.324572] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12302.346212] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12302.350094] LustreError: Skipped 1 previous similar message [12302.360706] Lustre: Skipped 1 previous similar message [12305.877894] Lustre: server umount lustre-OST0001 complete [12307.426138] LustreError: 328882:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12307.443286] LustreError: 328882:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12313.574442] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12313.588723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12313.613892] Lustre: Skipped 1 previous similar message [12313.624804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12317.664946] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12317.670874] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12317.677240] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12317.709047] Lustre: Skipped 1 previous similar message [12322.786610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12327.391731] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12327.596972] Lustre: server umount lustre-OST0000 complete [12332.105669] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12332.109767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12332.123349] Lustre: Skipped 1 previous similar message [12332.138669] Lustre: Skipped 2 previous similar messages [12338.637109] Lustre: server umount lustre-MDT0000 complete [12342.242295] LustreError: 326795:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12342.280896] LustreError: 326795:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [12342.983880] LustreError: 326776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787709447 with bad export cookie 3630273629455215905 [12342.999106] LustreError: 326776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12343.000191] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12343.395603] Lustre: server umount lustre-MDT0001 complete [12355.758310] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [12358.641744] Key type lgssc unregistered [12359.085343] LNet: 331711:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12359.111157] LNetError: 331711:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12359.130122] LNet: Removed LNI 192.168.204.149@tcp [12360.019975] Key type .llcrypt unregistered [12360.023992] Key type ._llcrypt unregistered [12402.889429] Key type ._llcrypt registered [12402.899369] Key type .llcrypt registered [12403.038102] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [12404.037791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12404.109490] alg: No test for adler32 (adler32-zlib) [12405.224438] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [12405.483930] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [12407.279191] Key type lgssc registered [12409.042287] Lustre: Echo OBD driver; http://www.lustre.org/ [12421.474753] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12421.502754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12423.316473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12428.648919] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12432.889269] LustreError: 333993:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12432.909606] LustreError: 333993:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12437.234050] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12437.505623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12442.062748] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12444.441881] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12450.839168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12451.339604] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12457.925605] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12462.643434] LustreError: 335477:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12466.653300] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12466.990246] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12468.040591] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12468.058382] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12473.328807] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12482.731151] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12489.434640] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12504.040688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12504.051262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12504.074653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12507.120733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12507.137246] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12507.163189] Lustre: Skipped 2 previous similar messages [12509.533609] Lustre: server umount lustre-MDT0000 complete [12512.229515] LustreError: 333989:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12512.263541] LustreError: 333989:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12513.538486] LustreError: 333973:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787709618 with bad export cookie 11143645610445882265 [12513.544158] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12513.553175] LustreError: 333973:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12513.973879] Lustre: server umount lustre-MDT0001 complete [12528.124565] Lustre: server umount lustre-OST0000 complete [12542.070530] Lustre: server umount lustre-OST0001 complete [12558.440605] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [12568.374351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12568.802701] LustreError: 339248:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12568.884360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12574.179814] LustreError: 339249:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12574.602577] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12584.428459] LustreError: 339248:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12584.458455] LustreError: 339248:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12585.431811] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12586.213724] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12593.660905] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12598.037253] Lustre: 340386:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12606.742622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12607.338964] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12610.428055] LustreError: 340739:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12613.512136] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12617.408647] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12628.321197] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12634.103603] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12636.122600] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12644.857967] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12650.693387] Lustre: 342250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12657.633249] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12657.639404] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12657.664504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12657.669874] Lustre: Skipped 3 previous similar messages [12659.681252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12659.689785] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12659.700747] Lustre: Skipped 2 previous similar messages [12663.649202] Lustre: server umount lustre-MDT0000 complete [12664.803243] LustreError: 339245:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12664.836563] LustreError: 339245:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [12669.440964] LustreError: 339229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787709774 with bad export cookie 11143645610445883966 [12669.463292] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12669.477335] LustreError: 339229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12669.924860] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12670.061760] Lustre: server umount lustre-MDT0001 complete [12676.740038] Lustre: server umount lustre-OST0000 complete [12681.547692] Lustre: server umount lustre-OST0001 complete [12685.632831] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12687.845244] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12690.076955] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:03:12 (1787709792) [12715.854515] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [12727.678933] loop0: detected capacity change from 0 to 1099511627776 [12728.304511] SGI XFS with ACLs, security attributes, quota, no debug enabled [12728.309503] *** VALIDATE xfs *** [12728.319927] XFS (loop0): Mounting V5 Filesystem [12728.339812] XFS (loop0): Ending clean mount [12730.152398] loop1: detected capacity change from 0 to 18691697672192 [12854.316711] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [12865.781726] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [12910.290900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12910.461767] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12910.480096] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12910.535879] Lustre: lustre-MDT0000: new disk, initializing [12910.651246] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12910.657074] Lustre: Skipped 1 previous similar message [12910.668448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12916.745296] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12928.201808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12928.351924] Lustre: 347356:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12928.473299] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12928.491117] Lustre: Skipped 1 previous similar message [12928.639124] Lustre: lustre-MDT0001: new disk, initializing [12928.901495] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12928.924254] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12933.634138] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12943.598774] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12949.469583] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12956.322288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12956.537826] Lustre: lustre-OST0000: new disk, initializing [12956.540873] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12956.545712] Lustre: 349317:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12957.970345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12957.986486] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12958.144484] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12962.748423] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [12972.871346] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12977.567936] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12977.803493] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12982.075352] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12982.295697] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12988.907540] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12988.909038] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12988.921496] LustreError: Skipped 1 previous similar message [12988.922260] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12988.922266] Lustre: Skipped 3 previous similar messages [12988.950726] Lustre: Skipped 2 previous similar messages [12990.828792] Lustre: server umount lustre-OST0000 complete [12995.552578] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12999.140207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12999.148037] Lustre: Skipped 3 previous similar messages [13001.015696] Lustre: server umount lustre-MDT0000 complete [13004.263210] LustreError: 347363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13004.303129] LustreError: 347363:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13005.229990] LustreError: 348149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787710109 with bad export cookie 11143645610445885520 [13005.234809] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13005.241027] LustreError: 348149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13005.570219] Lustre: server umount lustre-MDT0001 complete [13014.217542] XFS (loop0): Unmounting Filesystem [13018.944691] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:08:41 (1787710121) [13026.221944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13026.678306] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13026.683273] Lustre: Skipped 2 previous similar messages [13031.078292] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13039.976736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13045.455663] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13052.958486] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13059.692912] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13066.865909] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13073.985681] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13082.222471] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13094.892594] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13094.897484] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13094.921191] Lustre: Skipped 2 previous similar messages [13094.938535] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13101.005832] Lustre: server umount lustre-OST0000 complete [13107.169519] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13111.146278] Lustre: server umount lustre-MDT0000 complete [13115.721253] LustreError: 353704:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787710220 with bad export cookie 11143645610445886402 [13115.721988] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13115.732939] LustreError: 353704:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13116.462836] Lustre: server umount lustre-MDT0001 complete [13127.535986] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [13130.065742] Key type lgssc unregistered [13130.343881] LNet: 355485:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13130.357560] LNetError: 355485:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13130.381720] LNet: Removed LNI 192.168.204.149@tcp [13131.262175] Key type .llcrypt unregistered [13131.264081] Key type ._llcrypt unregistered [13163.167876] Key type ._llcrypt registered [13163.177972] Key type .llcrypt registered [13163.509996] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [13164.922075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13165.106732] alg: No test for adler32 (adler32-zlib) [13166.280990] Lustre: Lustre: Build Version: 2.17.57_80_g4b309f5 [13166.527431] LNet: Added LNI 192.168.204.149@tcp [8/256/0/180] [13168.175214] Key type lgssc registered [13169.285138] Lustre: Echo OBD driver; http://www.lustre.org/ [13176.569073] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13176.590578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13178.112965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13182.499591] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13187.556353] LustreError: 357050:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13187.587166] LustreError: 357050:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13192.687824] LustreError: 357049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13192.882941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13193.450868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13198.991375] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13209.293854] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13216.155446] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13225.420244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13226.099843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13235.021892] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13245.151949] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13258.222424] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13258.232731] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13258.244286] Lustre: Skipped 1 previous similar message [13258.259826] Lustre: Skipped 1 previous similar message [13261.105411] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.49@tcp (stopping) [13263.329831] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13263.342853] Lustre: Skipped 1 previous similar message [13263.498657] Lustre: server umount lustre-MDT0001 complete [13266.179604] LustreError: 357044:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13268.448813] LustreError: 357049:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13268.486816] LustreError: 357049:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13273.575370] LustreError: 357050:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13273.594586] LustreError: 357050:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [13275.533556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13275.626584] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13275.636398] Lustre: Found index 1 for lustre-MDT0001, updating log [13275.669817] Lustre: 357039:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13276.037746] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13281.269836] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13281.620361] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13284.386601] LustreError: 360616:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13284.391284] LustreError: 360616:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13284.401728] LustreError: 357041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13284.461000] LustreError: 360624:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.149@tcp: cfg command failed: rc = -17 [13284.469816] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13284.469816] [13291.494249] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13291.499027] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13291.509104] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13291.523337] Lustre: Skipped 1 previous similar message [13296.610614] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13296.619103] Lustre: Skipped 1 previous similar message [13297.683875] Lustre: server umount lustre-MDT0001 complete [13301.731257] LustreError: 357045:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13301.746782] LustreError: 357045:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [13309.248743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13309.338571] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13309.343670] Lustre: Found index 1 for lustre-MDT0001, updating log [13309.395215] Lustre: 357037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13309.718682] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13314.118102] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13315.058106] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13316.587765] LustreError: 361668:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13316.595754] LustreError: 361668:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13316.606678] LustreError: 357041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13316.620409] LustreError: 357041:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13316.718524] LustreError: 361677:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13316.736103] LustreError: 361677:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13316.743224] LustreError: 361677:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.149@tcp: cfg command failed: rc = -17 [13316.759030] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.149@tcp [13316.759030] [13619.484378] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.49@tcp (stopping) [13619.495623] Lustre: Skipped 1 previous similar message [13620.195446] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13620.221158] Lustre: Skipped 1 previous similar message [13625.622404] Lustre: server umount lustre-MDT0001 complete [13629.701473] LustreError: 363609:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13629.738543] LustreError: 363609:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [13637.055344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13637.230360] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13637.236324] Lustre: Found index 1 for lustre-MDT0001, updating log [13637.297289] Lustre: 357037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13637.745395] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13642.270221] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [13642.775072] LustreError: 369808:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13642.800596] LustreError: 369808:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13642.818619] LustreError: 357041:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13642.827879] LustreError: 357041:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13642.909698] LustreError: 369816:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13642.937302] LustreError: 369816:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.204.149@tcp: cfg command failed: rc = -114 [13642.954105] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13642.954105] [13643.245182] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13965.287952] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13965.293924] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13965.317058] Lustre: Skipped 1 previous similar message [13965.327589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13965.337829] Lustre: Skipped 5 previous similar messages [13969.871658] Lustre: server umount lustre-MDT0000 complete [13970.921894] LustreError: 357776:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13970.956234] LustreError: 357776:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [13973.684620] LustreError: 357783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787711078 with bad export cookie 3392366707251411080 [13973.685589] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13973.708942] LustreError: 357783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13976.033592] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13976.062581] Lustre: Skipped 2 previous similar messages [13988.319295] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13988.755867] Lustre: server umount lustre-MDT0001 complete [13993.163162] Lustre: server umount lustre-OST0000 complete [14002.030898] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [14012.568129] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [14050.297172] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:25:53 (1787711153) [14057.424878] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14057.666868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14057.688289] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14057.750619] Lustre: lustre-MDT0000: new disk, initializing [14057.807866] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14057.819942] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14062.525474] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [14073.405413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14073.575363] Lustre: 380366:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14073.647805] LustreError: 380375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14073.657879] LustreError: 380375:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [14073.659135] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14073.670239] Lustre: Skipped 1 previous similar message [14073.738717] Lustre: lustre-MDT0001: new disk, initializing [14073.854745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14073.920256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14073.938815] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14078.475804] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [14088.122687] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14093.897757] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14100.084609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14100.342190] Lustre: lustre-OST0000: new disk, initializing [14100.346694] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14100.353246] Lustre: 382327:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14100.469613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14105.668429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14105.676923] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14105.806679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14105.890474] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_default_debug -1 all [14114.120542] Lustre: DEBUG MARKER: oleg449-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14121.444692] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14121.447877] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14121.468744] Lustre: Skipped 2 previous similar messages [14121.474555] Lustre: Skipped 7 previous similar messages [14123.660379] Lustre: server umount lustre-MDT0000 complete [14126.928228] LustreError: 380357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787711231 with bad export cookie 3392366707251413033 [14126.928550] LustreError: 381173:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14126.931110] LustreError: MGC192.168.204.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14126.936220] LustreError: 380357:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14127.202816] Lustre: server umount lustre-MDT0001 complete [14141.190178] Lustre: server umount lustre-OST0000 complete [14168.952987] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing set_hostid [14178.271456] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing load_modules_local [14211.019764] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13921 sec ========= 22:28:33 (1787711313) [14212.732442] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:28:35 (1787711315) === [14216.158083] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:28:39 (1787711319) === [14258.007987] Lustre: DEBUG MARKER: oleg449-server.virtnet: executing unload_modules_local [14261.075327] Key type lgssc unregistered [14261.329746] LNet: 388673:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14261.342377] LNetError: 388673:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14261.369178] LNet: Removed LNI 192.168.204.149@tcp [14262.388141] Key type .llcrypt unregistered [14262.390685] Key type ._llcrypt unregistered