[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 421783010 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001008] APIC: Switch to symmetric I/O mode setup [ 0.002262] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004006] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007040] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008008] pid_max: default: 32768 minimum: 301 [ 0.009128] LSM: Security Framework initializing [ 0.010040] Yama: becoming mindful. [ 0.011031] SELinux: Initializing. [ 0.012072] *** VALIDATE selinux *** [ 0.020620] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024644] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025144] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026073] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027079] *** VALIDATE tmpfs *** [ 0.028295] *** VALIDATE proc *** [ 0.030048] *** VALIDATE cgroup *** [ 0.031005] *** VALIDATE cgroup2 *** [ 0.032229] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033133] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.035027] Spectre V2 : User space: Vulnerable [ 0.036006] Speculative Store Bypass: Vulnerable [ 0.039082] debug: unmapping init [mem 0xffffffffbd459000-0xffffffffbd460fff] [ 0.041776] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042699] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043020] ... version: 2 [ 0.044010] ... bit width: 48 [ 0.045008] ... generic registers: 4 [ 0.046008] ... value mask: 0000ffffffffffff [ 0.047009] ... max period: 00007fffffffffff [ 0.048016] ... fixed-purpose events: 3 [ 0.049015] ... event mask: 000000070000000f [ 0.050386] rcu: Hierarchical SRCU implementation. [ 0.052540] smp: Bringing up secondary CPUs ... [ 0.053566] x86: Booting SMP configuration: [ 0.054023] .... node #0, CPUs: #1 #2 #3 [ 0.059777] smp: Brought up 1 node, 4 CPUs [ 0.061011] smpboot: Max logical packages: 1 [ 0.062010] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.134652] node 0 deferred pages initialised in 69ms [ 0.138009] devtmpfs: initialized [ 0.139078] x86/mm: Memory block size: 128MB [ 0.141406] gcov: version magic: 0x41383552 [ 0.143217] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.145058] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.147246] pinctrl core: initialized pinctrl subsystem [ 0.148126] [ 0.148507] ************************************************************* [ 0.150008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.152007] ** ** [ 0.153006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.155007] ** ** [ 0.156006] ** This means that this kernel is built to expose internal ** [ 0.158006] ** IOMMU data structures, which may compromise security on ** [ 0.159006] ** your system. ** [ 0.161007] ** ** [ 0.162006] ** If you see this message and you are not debugging the ** [ 0.164007] ** kernel, report this immediately to your vendor! ** [ 0.165006] ** ** [ 0.167007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.168000] ************************************************************* [ 0.168256] NET: Registered protocol family 16 [ 0.169000] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.171053] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.173034] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.175089] cpuidle: using governor menu [ 0.176707] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.178407] PCI: Using configuration type 1 for base access [ 0.180083] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.187115] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.190018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.193067] cryptd: max_cpu_qlen set to 1000 [ 0.196158] ACPI: Added _OSI(Module Device) [ 0.197012] ACPI: Added _OSI(Processor Device) [ 0.198009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.199009] ACPI: Added _OSI(Processor Aggregator Device) [ 0.201000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.208365] ACPI: Interpreter enabled [ 0.210052] ACPI: PM: (supports S0 S3 S4 S5) [ 0.211009] ACPI: Using IOAPIC for interrupt routing [ 0.213107] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.216392] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.225233] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.227042] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.230017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.233085] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.238320] acpiphp: Slot [2] registered [ 0.240086] acpiphp: Slot [5] registered [ 0.242077] acpiphp: Slot [6] registered [ 0.243063] acpiphp: Slot [7] registered [ 0.244049] acpiphp: Slot [8] registered [ 0.245073] acpiphp: Slot [9] registered [ 0.247439] acpiphp: Slot [10] registered [ 0.250096] acpiphp: Slot [3] registered [ 0.251045] acpiphp: Slot [4] registered [ 0.252030] acpiphp: Slot [11] registered [ 0.253054] acpiphp: Slot [12] registered [ 0.254053] acpiphp: Slot [13] registered [ 0.255071] acpiphp: Slot [14] registered [ 0.256095] acpiphp: Slot [15] registered [ 0.257063] acpiphp: Slot [16] registered [ 0.259057] acpiphp: Slot [17] registered [ 0.260073] acpiphp: Slot [18] registered [ 0.261066] acpiphp: Slot [19] registered [ 0.262087] acpiphp: Slot [20] registered [ 0.263065] acpiphp: Slot [21] registered [ 0.265060] acpiphp: Slot [22] registered [ 0.266081] acpiphp: Slot [23] registered [ 0.267065] acpiphp: Slot [24] registered [ 0.268067] acpiphp: Slot [25] registered [ 0.270069] acpiphp: Slot [26] registered [ 0.271066] acpiphp: Slot [27] registered [ 0.272064] acpiphp: Slot [28] registered [ 0.273050] acpiphp: Slot [29] registered [ 0.273848] acpiphp: Slot [30] registered [ 0.275073] acpiphp: Slot [31] registered [ 0.276000] PCI host bridge to bus 0000:00 [ 0.276000] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.278017] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.280012] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.282015] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.284014] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.286014] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.287115] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.289720] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.290709] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.298011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.303572] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.305012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.307007] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.308011] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.311008] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.313222] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.315031] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.317787] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.322011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.334962] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.338012] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.342217] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.348032] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.355027] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.370019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.384623] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.395040] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.401015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.415017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.428097] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.433013] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.437014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.456022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.464546] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.470011] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.475016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.489015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.498025] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.505011] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.513012] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.543023] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.555101] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.562016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.569039] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.595019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.623569] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.625409] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.627369] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.629421] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.635273] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.640158] iommu: Default domain type: Passthrough [ 0.641000] SCSI subsystem initialized [ 0.641139] ACPI: bus type USB registered [ 0.642096] usbcore: registered new interface driver usbfs [ 0.644049] usbcore: registered new interface driver hub [ 0.645082] usbcore: registered new device driver usb [ 0.646154] pps_core: LinuxPPS API ver. 1 registered [ 0.648007] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.650053] PTP clock support registered [ 0.651191] EDAC MC: Ver: 3.0.0 [ 0.653111] PCI: Using ACPI for IRQ routing [ 0.654538] NetLabel: Initializing [ 0.656012] NetLabel: domain hash size = 128 [ 0.657009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.658098] NetLabel: unlabeled traffic allowed by default [ 0.661048] vgaarb: loaded [ 0.662283] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.663011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.668379] clocksource: Switched to clocksource kvm-clock [ 0.789973] VFS: Disk quotas dquot_6.6.0 [ 0.791421] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.793421] *** VALIDATE ramfs *** [ 0.794397] *** VALIDATE hugetlbfs *** [ 0.795637] pnp: PnP ACPI init [ 0.798175] pnp: PnP ACPI: found 6 devices [ 0.821453] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.824971] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.827679] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.830237] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.833408] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.835690] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.839205] NET: Registered protocol family 2 [ 0.844675] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.850015] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.853245] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.860333] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.864716] TCP: Hash tables configured (established 65536 bind 65536) [ 0.868458] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.871827] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.875666] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.879398] NET: Registered protocol family 1 [ 0.883513] RPC: Registered named UNIX socket transport module. [ 0.886325] RPC: Registered udp transport module. [ 0.889277] RPC: Registered tcp transport module. [ 0.892401] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.896096] NET: Registered protocol family 44 [ 0.899208] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.902367] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.906473] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.908963] PCI: CLS 0 bytes, default 64 [ 0.917258] Unpacking initramfs... [ 2.582695] debug: unmapping init [mem 0xffff9cc3bcc54000-0xffff9cc3bffbffff] [ 2.587107] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.589590] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.593065] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.147258] Initialise system trusted keyrings [ 3.149094] Key type blacklist registered [ 3.151282] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.161753] zbud: loaded [ 3.165264] *** VALIDATE nfs *** [ 3.166490] *** VALIDATE nfs4 *** [ 3.168120] pstore: using deflate compression [ 3.172165] Platform Keyring initialized [ 3.288566] NET: Registered protocol family 38 [ 3.292242] Key type asymmetric registered [ 3.293958] Asymmetric key parser 'x509' registered [ 3.296483] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.300597] io scheduler mq-deadline registered [ 3.303814] io scheduler kyber registered [ 3.305956] io scheduler bfq registered [ 3.308212] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.311947] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.314903] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.318582] ACPI: Power Button [PWRF] [ 3.326166] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.333597] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.351119] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.359115] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.377246] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.407231] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.433427] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.437678] Non-volatile memory driver v1.3 [ 3.439140] Linux agpgart interface v0.103 [ 3.472990] virtio_blk virtio1: [vda] 145800 512-byte logical blocks (74.6 MB/71.2 MiB) [ 3.476081] vda: detected capacity change from 0 to 74649600 [ 3.489823] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.493101] vdb: detected capacity change from 0 to 1073741824 [ 3.507474] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.510187] vdc: detected capacity change from 0 to 2621440000 [ 3.524783] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.527623] vdd: detected capacity change from 0 to 2621440000 [ 3.553087] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.557068] vde: detected capacity change from 0 to 4294967296 [ 3.574204] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.576934] vdf: detected capacity change from 0 to 4294967296 [ 3.585202] libphy: Fixed MDIO Bus: probed [ 3.591362] usbcore: registered new interface driver usbserial_generic [ 3.594160] usbserial: USB Serial support registered for generic [ 3.596851] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.601734] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.603194] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.606323] mousedev: PS/2 mouse device common for all mice [ 3.609533] rtc_cmos 00:05: RTC can wake from S4 [ 3.613389] rtc_cmos 00:05: registered as rtc0 [ 3.615185] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.618741] intel_pstate: CPU model not supported [ 3.623480] hid: raw HID events driver (C) Jiri Kosina [ 3.625157] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.626659] usbcore: registered new interface driver usbhid [ 3.626664] usbhid: USB HID core driver [ 3.626835] drop_monitor: Initializing network drop monitor service [ 3.628143] Initializing XFRM netlink socket [ 3.637076] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.641781] NET: Registered protocol family 10 [ 3.645770] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.658404] Segment Routing with IPv6 [ 3.659794] NET: Registered protocol family 17 [ 3.662197] mpls_gso: MPLS GSO support [ 3.667988] RAS: Correctable Errors collector initialized. [ 3.670030] AVX version of gcm_enc/dec engaged. [ 3.671211] AES CTR mode by8 optimization enabled [ 3.764633] sched_clock: Marking stable (3764545242, 0)->(4556808535, -792263293) [ 3.768941] registered taskstats version 1 [ 3.772802] Loading compiled-in X.509 certificates [ 3.775487] zswap: loaded using pool lzo/zbud [ 3.808617] Key type big_key registered [ 3.824858] Key type encrypted registered [ 3.826521] ima: No TPM chip found, activating TPM-bypass! [ 3.828604] ima: Allocated hash algorithm: sha1 [ 3.830187] ima: No architecture policies found [ 3.831653] evm: Initialising EVM extended attributes: [ 3.833482] evm: security.selinux [ 3.834556] evm: security.ima [ 3.835566] evm: security.capability [ 3.836695] evm: HMAC attrs: 0x1 [ 3.839490] rtc_cmos 00:05: setting system clock to 2026-07-15 05:20:35 UTC (1784092835) [ 3.846560] debug: unmapping init [mem 0xffffffffbe403000-0xffffffffbe5fffff] [ 3.850336] debug: unmapping init [mem 0xffffffffbd182000-0xffffffffbd458fff] [ 3.859376] Write protecting the kernel read-only data: 28672k [ 3.865688] debug: unmapping init [mem 0xffffffffbb803000-0xffffffffbb9fffff] [ 3.869095] debug: unmapping init [mem 0xffffffffbc114000-0xffffffffbc1fffff] [ 3.907231] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.915968] systemd[1]: Detected virtualization kvm. [ 3.918426] systemd[1]: Detected architecture x86-64. [ 3.920494] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.950544] systemd[1]: No hostname configured. [ 3.952439] systemd[1]: Set hostname to . [ 3.954815] random: systemd: uninitialized urandom read (16 bytes read) [ 3.957808] systemd[1]: Initializing machine ID from random generator. [ 4.037025] random: ln: uninitialized urandom read (6 bytes read) [ 4.178318] random: systemd: uninitialized urandom read (16 bytes read) [ 4.181427] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.185861] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.198965] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.516404] device-mapper: uevent: version 1.0.3 [ 5.519342] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.497331] virtio_net virtio0 ens2: renamed from eth0 [ 6.499167] random: fast init done [ 6.610496] scsi host0: ata_piix [ 6.776152] scsi host1: ata_piix [ 6.779081] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.785509] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.343775] random: crng init done [ 11.346451] random: 7 urandom warning(s) missed due to ratelimiting [ 12.627115] dracut-initqueue[586]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 14.380304] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 16.120908] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.677872] SELinux: Disabled at runtime. [ 16.767182] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.786550] systemd[1]: Detected virtualization kvm. [ 16.790346] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.779337] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.783538] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.794559] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.804359] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.813237] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.825266] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.842876] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Switch Root. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-serial\x2dgetty.slice. Mounting Huge Pages File System... [ OK ] Created slice User and Session Slice. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Reached target rpc_pipefs.target. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. [ 18.233305] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... [ OK ] Reached target Local Encrypted Volumes. Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 18.863969] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 19.732222] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 19.957868] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.317132] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 20.373567] EDAC sbridge: Ver: 1.1.2 [ 23.103394] Key type dns_resolver registered [ 23.768268] NFS: Registering the id_resolver key type [ 23.772976] Key type id_resolver registered [ 23.774598] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg202-server login: [ 61.934129] libcfs: loading out-of-tree module taints kernel. [ 61.959252] Key type ._llcrypt registered [ 61.960791] Key type .llcrypt registered [ 62.051081] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 73.208649] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 74.054446] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 74.064920] alg: No test for adler32 (adler32-zlib) [ 75.159222] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 75.631923] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 77.287339] Key type lgssc registered [ 78.170884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.931146] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 107.162080] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 113.327282] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 113.351459] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 114.479509] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 114.502028] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 114.548814] Lustre: lustre-MDT0000: new disk, initializing [ 114.591848] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 114.599965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 116.594564] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 123.414599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 123.456982] Lustre: 6488:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 123.470594] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 123.473479] Lustre: Skipped 1 previous similar message [ 123.493995] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 123.518597] Lustre: lustre-MDT0001: new disk, initializing [ 123.543633] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 123.557021] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 123.562409] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 125.246145] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 127.942355] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 132.238301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 132.389529] Lustre: lustre-OST0000: new disk, initializing [ 132.391667] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 132.394365] Lustre: 8425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 132.428822] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 135.157345] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 139.313951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 139.320636] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 139.340054] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 143.096188] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 143.173451] Lustre: lustre-OST0001: new disk, initializing [ 143.176814] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 143.182123] Lustre: 9495:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 143.225113] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 146.416617] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 147.964939] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 147.968214] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 147.989544] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 153.098460] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 158.330274] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 161.643735] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing check_logdir /tmp/testlogs/ [ 164.373692] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing yml_node [ 166.562318] Lustre: DEBUG MARKER: Client: 2.17.54.162 [ 167.893895] Lustre: DEBUG MARKER: MDS: 2.17.54.162 [ 169.171391] Lustre: DEBUG MARKER: OSS: 2.17.54.162 [ 170.017309] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Jul 15 01:23:20 EDT 2026 [ 178.661408] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 179.379853] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 194.018192] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 194.019572] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 194.026259] Lustre: Skipped 1 previous similar message [ 197.557243] Lustre: server umount lustre-MDT0000 complete [ 199.442905] LustreError: 6479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093031 with bad export cookie 9806573576110719588 [ 199.444493] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 199.448348] LustreError: 6479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 204.255729] LustreError: 6495:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 204.255865] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 204.256459] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 204.256464] Lustre: Skipped 3 previous similar messages [ 204.261426] LustreError: 6495:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 204.267285] Lustre: Skipped 3 previous similar messages [ 205.653637] Lustre: server umount lustre-MDT0001 complete [ 207.302556] Lustre: server umount lustre-OST0000 complete [ 208.965839] Lustre: server umount lustre-OST0001 complete [ 211.968979] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 215.857769] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 230.777540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 230.872715] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 230.888899] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 230.927463] Lustre: lustre-MDT0000: new disk, initializing [ 230.967073] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 230.975740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 232.582861] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 237.707354] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 237.756969] Lustre: 15037:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 237.888313] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 239.450337] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 243.514879] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 245.564130] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 248.179977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 248.291090] Lustre: lustre-OST0000: new disk, initializing [ 248.292633] Lustre: Skipped 1 previous similar message [ 248.294662] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 248.297129] Lustre: Skipped 2 previous similar messages [ 248.299798] Lustre: 17003:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 248.336500] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 248.338745] Lustre: Skipped 1 previous similar message [ 250.053114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 250.056319] Lustre: Skipped 1 previous similar message [ 250.059124] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 250.114286] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 250.692491] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 255.056742] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 256.797580] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 256.908509] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 258.659180] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 258.747535] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 260.579620] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 260.579838] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 260.581287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 260.581294] Lustre: Skipped 1 previous similar message [ 260.584278] Lustre: Skipped 1 previous similar message [ 260.586859] LustreError: Skipped 1 previous similar message [ 265.697202] Lustre: server umount lustre-OST0000 complete [ 268.768132] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 268.771993] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 268.778044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 268.780172] Lustre: Skipped 1 previous similar message [ 273.443529] Lustre: server umount lustre-MDT0000 complete [ 274.841247] LustreError: 15028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093106 with bad export cookie 9806573576110721128 [ 274.843870] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 274.846057] LustreError: 15028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 274.964090] Lustre: server umount lustre-MDT0001 complete [ 276.755979] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:25:07 (1784093107) [ 279.444637] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 279.617177] LustreError: 19266:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 279.641484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 281.213937] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 284.597344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 284.640181] LustreError: 19267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 286.286522] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 289.130630] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 291.164870] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 293.669925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 295.980822] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 298.885385] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 311.779184] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 315.363181] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 315.363906] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 315.364370] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 315.373582] Lustre: Skipped 3 previous similar messages [ 315.375653] Lustre: Skipped 3 previous similar messages [ 321.315964] Lustre: server umount lustre-MDT0000 complete [ 322.811597] LustreError: 19247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093154 with bad export cookie 9806573576110722010 [ 322.813036] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 322.817480] LustreError: 19247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 322.957726] Lustre: server umount lustre-MDT0001 complete [ 335.147662] Lustre: server umount lustre-OST0000 complete [ 344.046069] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 348.971464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 349.246406] LustreError: 23665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 349.288795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 349.291596] Lustre: Skipped 2 previous similar messages [ 351.313340] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 354.272868] LustreError: 23666:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 355.145126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 356.904261] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 358.137869] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 360.904180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 364.119984] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 369.090397] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 369.191404] Lustre: lustre-OST0001: new disk, initializing [ 369.196649] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 369.204031] Lustre: 25979:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 370.370216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 370.378437] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 370.449562] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 372.084674] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 376.941109] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 378.386699] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 383.970578] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 383.973818] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 383.976974] Lustre: Skipped 2 previous similar messages [ 383.979511] Lustre: Skipped 4 previous similar messages [ 389.786538] Lustre: server umount lustre-MDT0000 complete [ 391.244292] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093222 with bad export cookie 9806573576110722857 [ 391.247768] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 391.249167] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 391.381098] Lustre: server umount lustre-MDT0001 complete [ 402.437996] Lustre: server umount lustre-OST0000 complete [ 414.385441] Lustre: server umount lustre-OST0001 complete [ 421.327936] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 425.692474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 425.838313] LustreError: 28997:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 425.867655] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 425.870398] Lustre: Skipped 3 previous similar messages [ 427.310102] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 431.070592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 433.072530] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 434.522272] Lustre: 30144:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 437.754676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 440.657635] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 444.363951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 447.071928] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 455.703100] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 457.246597] Lustre: 32021:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 460.771645] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 460.772988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 460.776903] Lustre: Skipped 3 previous similar messages [ 460.780714] Lustre: Skipped 7 previous similar messages [ 465.387566] Lustre: server umount lustre-MDT0000 complete [ 465.889785] LustreError: 28998:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 465.895777] LustreError: 28998:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 466.902987] LustreError: 30147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093298 with bad export cookie 9806573576110724152 [ 466.905087] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 466.907493] LustreError: 30147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 467.032660] Lustre: server umount lustre-MDT0001 complete [ 479.244880] Lustre: server umount lustre-OST0000 complete [ 491.239688] Lustre: server umount lustre-OST0001 complete [ 495.015303] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:28:46 (1784093326) [ 506.813689] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 511.040537] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 511.204073] LustreError: 34818:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 513.064547] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 516.695545] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 518.636919] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 520.094193] Lustre: 35965:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 522.970740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 525.648794] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 529.285536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 531.905585] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 540.709383] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 542.154318] Lustre: 37841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 545.764478] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.765638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 545.770432] Lustre: Skipped 4 previous similar messages [ 545.774742] Lustre: Skipped 2 previous similar messages [ 549.852095] Lustre: server umount lustre-MDT0000 complete [ 551.372298] LustreError: 37843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093383 with bad export cookie 9806573576110725251 [ 551.375996] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 551.376851] LustreError: 37843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 551.495611] Lustre: server umount lustre-MDT0001 complete [ 563.207128] Lustre: server umount lustre-OST0000 complete [ 575.145483] Lustre: server umount lustre-OST0001 complete [ 578.503707] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:30:09 (1784093409) [ 584.708857] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 588.483621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 588.660752] LustreError: 39977:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 588.666774] LustreError: 39977:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 588.693031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 588.696305] Lustre: Skipped 7 previous similar messages [ 590.401837] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 594.007292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 595.803563] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 597.038232] Lustre: 41127:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 599.749520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 602.265293] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 606.155852] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 608.648441] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 617.517748] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 619.319686] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 619.323786] Lustre: Skipped 1 previous similar message [ 623.586980] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 623.592608] Lustre: Skipped 3 previous similar messages [ 626.669983] Lustre: server umount lustre-MDT0000 complete [ 628.220500] LustreError: 39960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093459 with bad export cookie 9806573576110726336 [ 628.225592] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 628.226884] LustreError: 39960:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 634.423190] Lustre: server umount lustre-MDT0001 complete [ 637.778588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 637.923883] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x8817f2b5c3cfe104 [ 637.928257] Lustre: MGC192.168.202.102@tcp: Connection restored to 0@lo (at 0@lo) [ 639.746204] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 643.713456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 645.708080] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 648.881272] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 656.643041] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 7 sec [ 657.382809] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 658.749826] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 659.361933] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 666.978557] Lustre: server umount lustre-MDT0000 complete [ 668.326552] LustreError: 45618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093499 with bad export cookie 9806573576110727428 [ 668.328519] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 668.332906] LustreError: 45618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 668.464386] Lustre: server umount lustre-MDT0001 complete [ 680.145371] Lustre: server umount lustre-OST0000 complete [ 692.205898] Lustre: server umount lustre-OST0001 complete [ 696.042059] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:32:07 (1784093527) [ 703.674825] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 708.117508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 709.962786] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 713.656578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 715.460242] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 716.566219] Lustre: 48757:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 716.569909] Lustre: 48757:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 719.151513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 721.532541] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 723.434453] LustreError: 49733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 723.442838] LustreError: 49733:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 724.827380] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 727.335960] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 736.130884] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 737.766983] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 743.395905] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 743.398272] Lustre: Skipped 19 previous similar messages [ 747.490138] Lustre: server umount lustre-MDT0000 complete [ 748.798115] LustreError: 50701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093580 with bad export cookie 9806573576110728240 [ 748.801684] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 748.804266] LustreError: 50701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 748.910311] Lustre: server umount lustre-MDT0001 complete [ 760.832807] Lustre: server umount lustre-OST0000 complete [ 771.427868] Lustre: server umount lustre-OST0001 complete [ 774.929555] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:33:26 (1784093606) [ 775.874361] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 776.569641] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:33:27 (1784093607) [ 780.880756] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 784.526530] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 793.500930] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 793.608629] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 793.647984] Lustre: lustre-MDT0000: new disk, initializing [ 793.700435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 795.286937] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 799.438654] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 801.959665] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 802.061676] Lustre: 54825:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 803.213573] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 803.230271] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 804.199722] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 808.429591] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 809.332224] Lustre: DEBUG MARKER: create test files [ 823.777178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 823.777190] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 823.784617] Lustre: Skipped 8 previous similar messages [ 827.102893] Lustre: server umount lustre-OST0000 complete [ 828.736450] Lustre: server umount lustre-MDT0000 complete [ 833.084655] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 834.240109] Key type lgssc unregistered [ 834.385428] LNet: 56449:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 834.388907] LNetError: 56449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 834.398665] LNet: Removed LNI 192.168.202.102@tcp [ 834.766165] Key type .llcrypt unregistered [ 834.767861] Key type ._llcrypt unregistered [ 846.118720] Key type ._llcrypt registered [ 846.120089] Key type .llcrypt registered [ 846.162775] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 846.572214] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 846.581997] alg: No test for adler32 (adler32-zlib) [ 847.441614] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 847.529500] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 849.111136] Key type lgssc registered [ 849.480394] Lustre: Echo OBD driver; http://www.lustre.org/ [ 852.063610] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 852.072401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 853.215988] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 854.582311] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 856.978525] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 859.324345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 859.432985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 861.358903] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 862.501302] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 862.527540] Lustre: 59155:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 862.530939] Lustre: 59155:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 862.533412] Lustre: 59155:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 862.535785] Lustre: 59155:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 862.538216] Lustre: 59155:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 862.540836] Lustre: 59155:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 863.902074] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 867.007532] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 868.891111] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 872.929575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 872.929698] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 872.935244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 877.016350] Lustre: server umount lustre-OST0000 complete [ 878.418804] Lustre: server umount lustre-MDT0000 complete [ 882.370241] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 883.407202] Key type lgssc unregistered [ 883.531386] LNet: 60760:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 883.533693] LNetError: 60760:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 883.542292] LNet: Removed LNI 192.168.202.102@tcp [ 883.862460] Key type .llcrypt unregistered [ 883.863764] Key type ._llcrypt unregistered [ 895.935459] Key type ._llcrypt registered [ 895.936674] Key type .llcrypt registered [ 895.982865] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 896.355117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 896.400804] alg: No test for adler32 (adler32-zlib) [ 897.253093] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 897.347964] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 898.935162] Key type lgssc registered [ 899.320826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 902.039403] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 902.048288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 903.205086] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 904.717755] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 907.695840] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 910.395101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 910.530196] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 912.613457] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 912.638318] Lustre: 63570:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 912.642534] Lustre: 63570:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 912.645115] Lustre: 63570:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 912.648080] Lustre: 63570:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 912.651115] Lustre: 63570:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 912.653762] Lustre: 63570:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 912.777813] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 915.625184] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 918.761961] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 923.105689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 923.105935] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 923.113405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 927.503865] Lustre: server umount lustre-OST0000 complete [ 929.040441] Lustre: server umount lustre-MDT0000 complete [ 933.055336] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 934.116620] Key type lgssc unregistered [ 934.238353] LNet: 65125:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 934.240930] LNetError: 65125:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 934.250282] LNet: Removed LNI 192.168.202.102@tcp [ 934.599321] Key type .llcrypt unregistered [ 934.601484] Key type ._llcrypt unregistered [ 941.061477] Key type ._llcrypt registered [ 941.063024] Key type .llcrypt registered [ 941.100527] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 947.921212] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 948.397326] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 948.420438] alg: No test for adler32 (adler32-zlib) [ 949.287205] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 949.390752] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 950.975144] Key type lgssc registered [ 951.369938] Lustre: Echo OBD driver; http://www.lustre.org/ [ 964.040434] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 964.047859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 965.136205] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 965.149055] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 965.186920] Lustre: lustre-MDT0000: new disk, initializing [ 965.210263] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 965.217087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 966.783363] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 971.215601] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 971.263203] Lustre: 68251:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 971.286861] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 971.288687] Lustre: Skipped 1 previous similar message [ 971.323140] Lustre: lustre-MDT0001: new disk, initializing [ 971.353275] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 971.365710] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 971.369371] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 972.779530] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 976.826230] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 978.967235] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 981.623156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 981.737284] Lustre: lustre-OST0000: new disk, initializing [ 981.739640] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 981.743138] Lustre: 70222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 981.772368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 984.026318] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 984.528754] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 984.534290] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 984.575394] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 988.159805] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 989.809497] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 989.888062] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 991.435753] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 991.511988] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 994.785707] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 994.785943] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 994.788379] LustreError: Skipped 1 previous similar message [ 994.789057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 994.792426] Lustre: Skipped 1 previous similar message [ 998.353659] Lustre: server umount lustre-OST0000 complete [ 999.904506] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 999.905179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 999.908796] Lustre: Skipped 1 previous similar message [ 999.911337] Lustre: Skipped 1 previous similar message [ 1005.024883] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1005.027124] Lustre: Skipped 2 previous similar messages [ 1005.787315] Lustre: server umount lustre-MDT0000 complete [ 1007.123372] LustreError: 68245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093838 with bad export cookie 9949087250253487787 [ 1007.125977] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1007.128960] LustreError: 68245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1007.260475] Lustre: server umount lustre-MDT0001 complete [ 1010.384169] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:37:21 (1784093841) [ 1014.086527] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 1027.829387] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 1031.367155] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 1044.348753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.426097] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1044.437336] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1044.473459] Lustre: lustre-MDT0000: new disk, initializing [ 1044.495101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1044.501681] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1045.891452] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1050.201583] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1050.244069] Lustre: 76227:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1050.267607] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1050.269446] Lustre: Skipped 1 previous similar message [ 1050.306464] Lustre: lustre-MDT0001: new disk, initializing [ 1050.337995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1050.349643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1050.353888] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1051.714164] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1055.935141] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1058.196387] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1060.837351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1060.938591] Lustre: lustre-OST0000: new disk, initializing [ 1060.940530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1060.944316] Lustre: 78196:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1060.980220] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1062.999370] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1066.986916] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1068.546764] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1070.077120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1070.080812] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1070.124516] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1070.663301] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 1072.168775] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1072.255964] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1075.170133] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1075.170896] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1075.174562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1075.178237] Lustre: Skipped 1 previous similar message [ 1079.125173] Lustre: server umount lustre-OST0000 complete [ 1081.311770] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1081.314339] LustreError: Skipped 1 previous similar message [ 1081.316031] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1081.320892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1081.322699] Lustre: Skipped 1 previous similar message [ 1085.408795] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1086.559334] Lustre: server umount lustre-MDT0000 complete [ 1087.865073] LustreError: 76218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784093919 with bad export cookie 9949087250253488655 [ 1087.867707] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1087.868575] LustreError: 76218:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1087.992782] Lustre: server umount lustre-MDT0001 complete [ 1090.969459] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 01:38:42 (1784093922) [ 1093.166194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1093.326363] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1094.762686] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1097.582815] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1099.077562] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1101.706284] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1103.565235] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1105.885147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1107.913570] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1110.389906] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1111.626784] Lustre: 83288:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 4c8beddc-1f58-4846-921d-8f0122c52d4a at adminstrative request [ 1113.570626] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1113.574137] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1123.810460] Lustre: MGS: Client 4c8beddc-1f58-4846-921d-8f0122c52d4a (at 0@lo) reconnecting [ 1123.813081] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8a1242243eba2573 to 0x8a1242243eba27fe [ 1123.818028] Lustre: MGC192.168.202.102@tcp: Connection restored to 0@lo (at 0@lo) [ 1143.964984] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1144.035927] Lustre: lustre-OST0001: new disk, initializing [ 1144.037581] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1144.040520] Lustre: 83679:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1144.068851] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1144.071453] Lustre: Skipped 2 previous similar messages [ 1146.004971] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1149.620612] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1150.584178] Lustre: server umount lustre-OST0001 complete [ 1153.003659] LustreError: 83301:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1153.006970] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1153.009347] LustreError: 83301:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1153.014531] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1153.016547] Lustre: Skipped 3 previous similar messages [ 1158.114281] LustreError: 84725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1158.118411] LustreError: 84725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1163.233922] LustreError: 82440:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1163.241131] LustreError: 82440:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1166.303106] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1166.357338] Lustre: server umount lustre-OST0000 complete [ 1173.473034] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1173.475865] Lustre: Skipped 7 previous similar messages [ 1173.720760] Lustre: server umount lustre-MDT0000 complete [ 1175.026228] LustreError: 81353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784094006 with bad export cookie 9949087250253490174 [ 1175.029230] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1175.029621] LustreError: 81353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1175.145251] Lustre: server umount lustre-MDT0001 complete [ 1177.921529] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 01:40:09 (1784094009) [ 1178.432944] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1179.016310] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 01:40:10 (1784094010) [ 1179.531900] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1180.126434] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 01:40:11 (1784094011) [ 1180.683989] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1181.219963] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 01:40:12 (1784094012) [ 1181.766281] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1182.331991] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 01:40:13 (1784094013) [ 1182.857241] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1184.962102] loop: module loaded [ 1184.967536] loop0: detected capacity change from 0 to 4294967296 [ 1184.990671] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1185.530308] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1186.615515] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1188.008460] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1196.921712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1196.940770] LustreError: 87789:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1196.943884] LustreError: 87789:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1196.946769] LustreError: 87789:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1196.949727] LustreError: 87789:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1203.250224] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 1206.578295] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 1218.986885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1219.066587] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1219.076454] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1219.112261] Lustre: lustre-MDT0000: new disk, initializing [ 1219.132354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1219.138281] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1220.514276] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1224.997784] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1225.042448] Lustre: 90648:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1225.177779] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1226.534521] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1230.456321] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1232.380729] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1234.761376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1234.869773] Lustre: 92614:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1235.054468] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1235.089019] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1237.130416] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1241.088055] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1242.639613] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1242.716663] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1244.247548] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1244.336690] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1245.670347] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1245.671168] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1245.672397] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1245.676030] Lustre: Skipped 4 previous similar messages [ 1251.089475] Lustre: server umount lustre-OST0000 complete [ 1255.906384] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1255.910137] LustreError: Skipped 1 previous similar message [ 1258.582026] Lustre: server umount lustre-MDT0000 complete [ 1259.871330] LustreError: 90640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784094091 with bad export cookie 9949087250253490755 [ 1259.874703] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1259.877174] LustreError: 90640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1259.988670] Lustre: server umount lustre-MDT0001 complete [ 1263.264101] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 01:41:34 (1784094094) [ 1266.365729] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1266.591442] LustreError: 95019:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1269.970285] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1271.778416] LustreError: 95020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1278.793721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1284.091193] hrtimer: interrupt took 9444149 ns [ 1286.449966] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1296.249152] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1302.130692] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1308.548598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1316.363963] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1322.819722] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1327.300982] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1327.527299] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1331.203317] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1331.345851] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1336.761876] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1336.885827] Lustre: lustre-OST0001: new disk, initializing [ 1336.888499] Lustre: Skipped 2 previous similar messages [ 1336.891696] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1336.894755] Lustre: Skipped 3 previous similar messages [ 1336.898451] Lustre: 98389:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1341.282506] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1341.995891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1342.014849] Lustre: Skipped 2 previous similar messages [ 1342.019462] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1342.148619] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1348.396417] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1351.841772] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1352.003709] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1355.228959] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1355.389161] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1371.100564] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1390.033797] Lustre: Failing over lustre-MDT0000 [ 1390.325738] Lustre: server umount lustre-MDT0000 complete [ 1392.097805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1392.105332] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1392.114115] Lustre: Skipped 2 previous similar messages [ 1392.117830] LustreError: 95020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1392.128663] LustreError: 95020:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1408.992722] LustreError: 95748:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1409.009910] LustreError: 95748:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 21 previous similar messages [ 1409.453955] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1409.456068] LDISKFS-fs (dm-0): recovery complete [ 1409.461720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1409.550154] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1409.693846] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1409.697714] Lustre: Skipped 6 previous similar messages [ 1409.728746] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1412.299507] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1414.551158] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1414.552212] Lustre: 99062:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1415.138392] Lustre: 95019:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1415.144309] Lustre: 95019:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 1415.152284] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1415.197605] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1415.511143] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1415.834264] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1415.841840] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1416.159160] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1416.165886] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1416.172954] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1416.183382] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 1417.146979] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1417.154691] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1417.471174] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1417.484757] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1418.443195] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1418.458213] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1419.416839] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1419.427277] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1419.745762] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1419.750633] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1422.576183] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1422.579893] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 1423.536350] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1423.544388] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1423.863083] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1423.868613] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1430.760498] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1430.767932] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 1431.713667] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1431.722044] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1432.039647] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1432.045428] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1446.890176] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1446.900755] Lustre: 101232:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 1447.856800] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1447.866191] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1448.183135] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1448.190220] LustreError: 101232:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 1474.007221] Lustre: 101232:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1474.017096] Lustre: 101232:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1474.027473] Lustre: 101232:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b0b4260e-645a-439b-90eb-b3eaa26de401@192.168.202.2@tcp [ 1474.036108] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1474.041164] Lustre: 101232:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@000000008003411f x1870757913237632/t0(8589934783) o101->b0b4260e-645a-439b-90eb-b3eaa26de401@192.168.202.2@tcp:626/0 lens 592/0 e 0 to 0 dl 1784094316 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1474.065372] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1474.073970] Lustre: lustre-MDT0000: Denying connection for new client b0b4260e-645a-439b-90eb-b3eaa26de401 (at 192.168.202.2@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1474.080927] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1474.117098] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1474.122351] Lustre: Skipped 2 previous similar messages [ 1474.130049] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1474.160302] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 1474.161194] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1481.702391] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1481.713082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1481.713091] Lustre: Skipped 5 previous similar messages [ 1481.725121] Lustre: Skipped 4 previous similar messages [ 1487.461450] Lustre: server umount lustre-OST0000 complete [ 1490.913704] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1491.942443] LustreError: 96843:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1491.949330] LustreError: 96843:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 1496.248864] Lustre: server umount lustre-OST0001 complete [ 1504.744171] Lustre: server umount lustre-MDT0000 complete [ 1506.905961] LustreError: 94999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784094338 with bad export cookie 9949087250253541953 [ 1506.908786] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1506.915566] LustreError: 94999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1507.111129] Lustre: server umount lustre-MDT0001 complete [ 1512.097860] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 01:45:43 (1784094343) [ 1516.604890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1516.608496] Lustre: *** cfs_fail_loc=197, val=0*** [ 1532.127681] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1537.247490] LustreError: 103205:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.102@tcp: failed processing log, type 4: rc = -110 [ 1542.388716] Lustre: *** cfs_fail_loc=197, val=0*** [ 1542.390778] Lustre: Skipped 65 previous similar messages [ 1566.879835] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1571.340612] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1572.625654] Lustre: server umount lustre-OST0000 complete [ 1577.631119] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 01:46:48 (1784094408) [ 1594.747611] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1595.044247] LustreError: 104825:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1598.621848] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1610.174138] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1615.682864] LustreError: 104856:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1615.710281] LustreError: 104856:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1615.732621] LustreError: 104856:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 1616.265953] Lustre: server umount lustre-MDT0000 complete [ 1662.762307] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 1683.551465] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 1721.618126] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1721.862895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1721.886778] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1721.960641] Lustre: lustre-MDT0000: new disk, initializing [ 1722.022366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1722.027168] Lustre: Skipped 2 previous similar messages [ 1722.037490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1726.365647] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1736.744156] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1736.834567] Lustre: 109054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1737.064737] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1741.610495] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1750.104874] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1755.803456] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1762.292554] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1762.543621] Lustre: 111021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1768.370595] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1772.815644] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1778.141323] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1783.503231] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1783.680610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1789.691544] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1789.928356] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1793.508184] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1793.525816] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1793.526369] Lustre: Skipped 4 previous similar messages [ 1793.529327] LustreError: Skipped 1 previous similar message [ 1793.530683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1793.530688] Lustre: Skipped 9 previous similar messages [ 1806.303186] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1806.536542] Lustre: server umount lustre-OST0000 complete [ 1817.327309] Lustre: server umount lustre-MDT0000 complete [ 1819.106261] LustreError: 109067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1819.118447] LustreError: 109067:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 1821.547376] LustreError: 109047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784094653 with bad export cookie 9949087250253552628 [ 1821.550713] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1821.563597] LustreError: 109047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1822.041037] Lustre: server umount lustre-MDT0001 complete [ 1831.264309] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 01:51:01 (1784094661) [ 1841.229250] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 1844.560667] Key type lgssc unregistered [ 1844.951749] LNet: 113510:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1844.960274] LNetError: 113510:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1845.002294] LNet: Removed LNI 192.168.202.102@tcp [ 1846.041916] Key type .llcrypt unregistered [ 1846.047058] Key type ._llcrypt unregistered [ 1864.315487] Key type ._llcrypt registered [ 1864.320235] Key type .llcrypt registered [ 1864.500822] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 1881.512556] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 1883.477054] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1883.505732] alg: No test for adler32 (adler32-zlib) [ 1884.624378] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 1884.894284] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 1886.575173] Key type lgssc registered [ 1887.749308] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1928.667761] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1928.719940] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1930.009236] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1930.031275] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1930.131169] Lustre: lustre-MDT0000: new disk, initializing [ 1930.233593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1930.260917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1935.039604] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1945.745616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1945.978483] Lustre: lustre-OST0000: new disk, initializing [ 1945.981781] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1945.985747] Lustre: Skipped 1 previous similar message [ 1945.989804] Lustre: 117691:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1946.059623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1951.829613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1951.836675] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1951.880859] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1952.986301] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1963.447995] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1963.667230] Lustre: lustre-OST0001: new disk, initializing [ 1963.671356] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1963.678578] Lustre: 118555:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1963.765698] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1970.555959] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 1973.932857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1973.944672] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1973.988231] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1984.481987] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1984.492261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1984.504744] Lustre: Skipped 1 previous similar message [ 1986.939798] Lustre: server umount lustre-MDT0000 complete [ 1993.322228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1998.017852] LustreError: 116878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784094829 with bad export cookie 7242143344101056115 [ 1998.023531] LustreError: 116878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1998.027394] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1998.154425] Lustre: server umount lustre-OST0000 complete [ 2001.693785] Lustre: server umount lustre-OST0001 complete [ 2009.828586] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 01:54:00 (1784094840) [ 2031.050763] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 01:54:21 (1784094861) [ 2041.744579] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2063.154568] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 01:54:53 (1784094893) [ 2099.855532] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 2110.634605] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2140.050882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2140.282063] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2140.312443] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2140.388523] Lustre: lustre-MDT0000: new disk, initializing [ 2140.447791] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2140.471107] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2144.765971] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2153.612976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2153.770588] Lustre: 126304:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2153.819369] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2153.822674] Lustre: Skipped 1 previous similar message [ 2153.880995] Lustre: lustre-MDT0001: new disk, initializing [ 2153.947607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2153.977823] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2153.991994] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2158.029662] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2166.784484] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2171.818539] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2177.896241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2178.151930] Lustre: lustre-OST0000: new disk, initializing [ 2178.156180] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2178.165436] Lustre: 128271:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2178.241438] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2184.410784] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2186.869064] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2186.878048] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2186.990460] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2193.373869] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2198.010440] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2198.234268] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2202.273849] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2202.419761] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2207.718938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2207.719531] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2207.729262] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2207.744380] Lustre: Skipped 2 previous similar messages [ 2207.745770] LustreError: Skipped 1 previous similar message [ 2210.266978] Lustre: server umount lustre-OST0000 complete [ 2215.392840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2215.404663] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2215.419741] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2215.424995] Lustre: Skipped 1 previous similar message [ 2217.952584] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2217.954230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2217.984512] Lustre: Skipped 1 previous similar message [ 2220.220711] Lustre: server umount lustre-MDT0000 complete [ 2223.507924] LustreError: 128270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095055 with bad export cookie 7242143344101057340 [ 2223.514127] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2223.522456] LustreError: 128270:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2223.779889] Lustre: server umount lustre-MDT0001 complete [ 2230.813963] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 01:57:41 (1784095061) [ 2235.901945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2236.289502] LustreError: 130683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2236.301703] LustreError: 130683:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2236.359744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2240.289485] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2241.504696] LustreError: 130684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2245.601616] LustreError: 130683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2247.412232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2251.587885] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2257.984660] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2262.918579] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2269.190928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2269.413234] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2269.420324] Lustre: Skipped 1 previous similar message [ 2275.985769] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2282.042158] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2286.510787] Lustre: *** cfs_fail_loc=159, val=0*** [ 2286.524096] Lustre: Skipped 1 previous similar message [ 2302.900703] Lustre: lustre-MDT0000: Client 55d579d2-5096-460f-8cfa-3bf89f776633 (at 192.168.202.2@tcp) reconnecting [ 2304.260043] Lustre: *** cfs_fail_loc=159, val=0*** [ 2304.266641] Lustre: Skipped 3 previous similar messages [ 2319.841255] Lustre: lustre-MDT0000: Client 55d579d2-5096-460f-8cfa-3bf89f776633 (at 192.168.202.2@tcp) reconnecting [ 2326.192851] Lustre: server umount lustre-OST0000 complete [ 2332.130501] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2332.135531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2332.147682] Lustre: Skipped 1 previous similar message [ 2332.166267] Lustre: Skipped 1 previous similar message [ 2340.368355] Lustre: server umount lustre-MDT0000 complete [ 2342.368638] Lustre: 114898:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784095158/real 1784095158] req@000000007cc67c65 x1870758901029376/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1784095174 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2342.372729] LustreError: 133403:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2342.385619] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2342.410621] LustreError: 133403:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2343.115770] LustreError: 130666:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095174 with bad export cookie 7242143344101058222 [ 2343.127140] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2343.377600] Lustre: server umount lustre-MDT0001 complete [ 2352.475977] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 2354.624722] Key type lgssc unregistered [ 2354.803806] LNet: 134457:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2354.809279] LNetError: 134457:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2354.818575] LNet: Removed LNI 192.168.202.102@tcp [ 2355.409937] Key type .llcrypt unregistered [ 2355.411645] Key type ._llcrypt unregistered [ 2375.629684] Key type ._llcrypt registered [ 2375.632905] Key type .llcrypt registered [ 2375.735098] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2376.516180] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2376.654818] alg: No test for adler32 (adler32-zlib) [ 2377.726895] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 2377.955828] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 2379.631173] Key type lgssc registered [ 2380.804863] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2387.875764] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2387.894234] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2389.394110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2393.685746] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2399.721789] LustreError: 135853:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2399.734206] LustreError: 135853:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2401.507587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2401.813415] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2405.123371] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2411.407513] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2416.191831] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2422.409069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2422.720380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2427.735284] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2434.279205] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2434.988837] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2443.157427] Lustre: *** cfs_fail_loc=159, val=0*** [ 2458.624938] Lustre: lustre-MDT0000: Client 43eebd23-908a-4ef5-92bb-2b0bb5752433 (at 192.168.202.2@tcp) reconnecting [ 2460.761213] Lustre: *** cfs_fail_loc=159, val=0*** [ 2477.009963] Lustre: lustre-MDT0001: Client 43eebd23-908a-4ef5-92bb-2b0bb5752433 (at 192.168.202.2@tcp) reconnecting [ 2478.116051] Lustre: *** cfs_fail_loc=159, val=0*** [ 2478.119938] Lustre: Skipped 3 previous similar messages [ 2493.362271] Lustre: lustre-MDT0001: Client 43eebd23-908a-4ef5-92bb-2b0bb5752433 (at 192.168.202.2@tcp) reconnecting [ 2496.391601] Lustre: 138907:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2498.270953] Lustre: *** cfs_fail_loc=159, val=0*** [ 2498.272809] Lustre: Skipped 4 previous similar messages [ 2514.875840] Lustre: lustre-MDT0000: Client ad6803ed-4be5-4ede-be80-0520e2014081 (at 192.168.202.2@tcp) reconnecting [ 2516.275357] Lustre: *** cfs_fail_loc=159, val=0*** [ 2516.279311] Lustre: Skipped 14 previous similar messages [ 2532.788117] Lustre: lustre-MDT0000: Client ad6803ed-4be5-4ede-be80-0520e2014081 (at 192.168.202.2@tcp) reconnecting [ 2534.167187] Lustre: 139201:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2552.802685] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2552.808311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2552.822755] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2554.852384] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2554.859755] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2554.867071] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2555.876928] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2557.532704] Lustre: server umount lustre-OST0000 complete [ 2560.482786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2560.488257] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2560.498738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2560.502078] Lustre: Skipped 1 previous similar message [ 2566.113392] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2566.118386] Lustre: Skipped 2 previous similar messages [ 2574.303468] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2574.422626] Lustre: server umount lustre-MDT0000 complete [ 2576.356393] LustreError: 139085:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2576.363045] LustreError: 139085:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2577.317146] LustreError: 135834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095408 with bad export cookie 6928933973570991390 [ 2577.320168] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2577.326336] LustreError: 135834:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2577.551630] Lustre: server umount lustre-MDT0001 complete [ 2585.380081] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 2587.093225] Key type lgssc unregistered [ 2587.235310] LNet: 140280:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2587.242106] LNetError: 140280:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2587.255739] LNet: Removed LNI 192.168.202.102@tcp [ 2587.873216] Key type .llcrypt unregistered [ 2587.875309] Key type ._llcrypt unregistered [ 2605.726108] Key type ._llcrypt registered [ 2605.728464] Key type .llcrypt registered [ 2605.833655] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2606.500443] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2606.528331] alg: No test for adler32 (adler32-zlib) [ 2607.517678] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 2607.701823] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 2609.351178] Key type lgssc registered [ 2610.153556] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2615.821190] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2615.833676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2617.149912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2620.606549] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2626.542405] LustreError: 141689:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2626.559021] LustreError: 141689:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2627.722219] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2628.013403] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2631.109916] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2635.943527] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2640.081800] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2644.965622] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2645.196389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2649.447604] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2652.403032] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2654.445113] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2661.628348] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2670.450772] Lustre: 144293:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2672.397690] Lustre: 144341:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2673.410984] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2674.077138] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 2676.346409] Lustre: 144487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2677.064055] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2677.833945] Lustre: 144585:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2682.345143] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2682.347696] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2682.360172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2682.369882] Lustre: Skipped 1 previous similar message [ 2682.376381] Lustre: Skipped 1 previous similar message [ 2687.134522] Lustre: server umount lustre-OST0000 complete [ 2692.589294] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2692.596576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2692.602663] Lustre: Skipped 1 previous similar message [ 2696.791403] Lustre: server umount lustre-MDT0000 complete [ 2697.700653] LustreError: 142418:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2697.724399] LustreError: 142418:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2699.682872] LustreError: 142425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095531 with bad export cookie 14640867622652071327 [ 2699.685770] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2699.692205] LustreError: 142425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2699.892702] Lustre: server umount lustre-MDT0001 complete [ 2707.036591] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 2709.272647] Key type lgssc unregistered [ 2709.561955] LNet: 145646:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2709.566924] LNetError: 145646:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2709.582486] LNet: Removed LNI 192.168.202.102@tcp [ 2710.319164] Key type .llcrypt unregistered [ 2710.325868] Key type ._llcrypt unregistered [ 2728.881513] Key type ._llcrypt registered [ 2728.883650] Key type .llcrypt registered [ 2728.957976] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2729.765639] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2729.780389] alg: No test for adler32 (adler32-zlib) [ 2730.752958] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 2730.936530] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 2732.583181] Key type lgssc registered [ 2733.365685] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2738.597467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2738.619440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2739.938918] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2742.857621] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2748.630485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2748.856447] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2751.718400] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2756.983477] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2761.028652] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2766.064584] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2766.422607] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2770.857321] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2773.614504] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2776.135356] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2784.875716] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2791.202542] Lustre: *** cfs_fail_loc=159, val=0*** [ 2806.717948] Lustre: lustre-MDT0000: Client 4fbd070c-1ea6-470b-99fd-d81556276a12 (at 192.168.202.2@tcp) reconnecting [ 2825.186345] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2825.194622] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2825.211287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2826.208343] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2826.214588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2826.222992] Lustre: Skipped 1 previous similar message [ 2829.531740] Lustre: server umount lustre-OST0000 complete [ 2835.936454] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2835.946090] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2835.959642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2838.385319] Lustre: server umount lustre-MDT0000 complete [ 2840.540462] LustreError: 147020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095672 with bad export cookie 7976538131843439490 [ 2840.541302] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2840.546735] LustreError: 147020:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2840.740085] Lustre: server umount lustre-MDT0001 complete [ 2846.882396] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 2848.417988] Key type lgssc unregistered [ 2848.605379] LNet: 150721:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2848.609069] LNetError: 150721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2848.617252] LNet: Removed LNI 192.168.202.102@tcp [ 2849.059909] Key type .llcrypt unregistered [ 2849.061555] Key type ._llcrypt unregistered [ 2863.256576] Key type ._llcrypt registered [ 2863.258060] Key type .llcrypt registered [ 2863.320704] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 2863.857367] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2863.871062] alg: No test for adler32 (adler32-zlib) [ 2864.815406] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 2864.965873] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 2866.591162] Key type lgssc registered [ 2867.260657] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2872.670411] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2872.685835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2873.961962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2877.254110] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2883.383876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2883.636642] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2886.307823] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2891.546420] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2895.503437] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2900.088849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2900.304701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2904.877536] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 2908.526799] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2910.598498] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2918.775539] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2938.071125] Lustre: lustre-MDT0001: evicting 84c320d1-daf5-4e70-a0cd-25580e84b7c7 (at 192.168.202.2@tcp) by administrative request [ 2938.079511] Lustre: Skipped 1 previous similar message [ 2964.960490] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2964.965552] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2964.970093] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2964.974605] Lustre: Skipped 1 previous similar message [ 2970.087336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2970.091179] Lustre: Skipped 2 previous similar messages [ 2970.666781] Lustre: server umount lustre-OST0000 complete [ 2975.200826] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2975.210186] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2979.319285] Lustre: server umount lustre-MDT0000 complete [ 2980.320548] LustreError: 152109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2980.330638] LustreError: 152109:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2981.682236] LustreError: 152855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095813 with bad export cookie 13104926266187532216 [ 2981.688078] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2981.692056] LustreError: 152855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2981.886395] Lustre: server umount lustre-MDT0001 complete [ 2989.210909] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 2990.864571] Key type lgssc unregistered [ 2991.052820] LNet: 156203:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2991.056871] LNetError: 156203:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2991.074535] LNet: Removed LNI 192.168.202.102@tcp [ 2991.526552] Key type .llcrypt unregistered [ 2991.527831] Key type ._llcrypt unregistered [ 3034.434409] Key type ._llcrypt registered [ 3034.435741] Key type .llcrypt registered [ 3034.492361] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 3035.000778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3035.007910] alg: No test for adler32 (adler32-zlib) [ 3035.915604] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 3036.030305] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 3037.631172] Key type lgssc registered [ 3038.204790] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3042.326235] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3042.337896] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3043.597929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3046.257643] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3051.215593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3051.421822] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3054.023183] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3058.862576] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3062.395661] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3066.344688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3066.603513] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3068.658647] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3070.211409] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3074.732607] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3076.848119] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3082.213350] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3082.213857] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3082.225904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3082.983217] Lustre: server umount lustre-OST0000 complete [ 3085.513605] Lustre: server umount lustre-MDT0000 complete [ 3087.330151] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3087.330882] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3087.336666] LustreError: 158667:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3087.342170] Lustre: Skipped 2 previous similar messages [ 3087.350637] LustreError: 158667:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3088.010746] LustreError: 158647:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784095919 with bad export cookie 14509003636413098434 [ 3088.013573] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3088.019145] LustreError: 158647:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3088.292971] Lustre: server umount lustre-MDT0001 complete [ 3094.973770] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 3096.619165] Key type lgssc unregistered [ 3096.806529] LNet: 162202:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3096.810782] LNetError: 162202:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3096.819382] LNet: Removed LNI 192.168.202.102@tcp [ 3097.344240] Key type .llcrypt unregistered [ 3097.345936] Key type ._llcrypt unregistered [ 3112.706744] Key type ._llcrypt registered [ 3112.711804] Key type .llcrypt registered [ 3112.787739] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 3113.384401] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3113.446075] alg: No test for adler32 (adler32-zlib) [ 3114.422949] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 3114.585094] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 3116.207708] Key type lgssc registered [ 3116.989772] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3126.266501] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 02:12:37 (1784095957) [ 3136.043184] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 3143.360324] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 3166.064690] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3166.076992] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3167.200100] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3167.227716] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3167.274200] Lustre: lustre-MDT0000: new disk, initializing [ 3167.325046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3167.337891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3169.989026] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3176.605459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3176.697421] Lustre: 166438:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3176.753433] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3176.755604] Lustre: Skipped 1 previous similar message [ 3176.807963] Lustre: lustre-MDT0001: new disk, initializing [ 3176.866101] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3176.888902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3176.894142] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3179.506533] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3185.240595] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3187.973462] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3191.551946] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3191.707228] Lustre: lustre-OST0000: new disk, initializing [ 3191.709337] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3191.713500] Lustre: 168405:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3191.769263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3194.577438] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3198.983406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3198.988230] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3199.047272] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3200.191303] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3209.184413] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3209.186864] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3209.191800] LustreError: Skipped 1 previous similar message [ 3209.195069] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3209.198023] Lustre: Skipped 1 previous similar message [ 3209.208342] Lustre: Skipped 1 previous similar message [ 3212.001552] Lustre: server umount lustre-OST0000 complete [ 3214.305372] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3214.318127] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3214.321502] Lustre: Skipped 1 previous similar message [ 3219.425868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3219.430448] Lustre: Skipped 1 previous similar message [ 3220.154755] Lustre: server umount lustre-MDT0000 complete [ 3222.030733] LustreError: 166430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784096053 with bad export cookie 9824085714843102207 [ 3222.033458] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3222.036679] LustreError: 166430:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3222.194156] Lustre: server umount lustre-MDT0001 complete [ 3227.195208] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 3228.596952] Key type lgssc unregistered [ 3228.746476] LNet: 170307:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3228.749966] LNetError: 170307:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3228.761621] LNet: Removed LNI 192.168.202.102@tcp [ 3229.206184] Key type .llcrypt unregistered [ 3229.208087] Key type ._llcrypt unregistered [ 3243.731601] Key type ._llcrypt registered [ 3243.733343] Key type .llcrypt registered [ 3243.785376] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 3244.255993] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3244.295548] alg: No test for adler32 (adler32-zlib) [ 3245.214880] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 3245.347340] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 3246.953222] Key type lgssc registered [ 3247.570799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3251.808425] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3251.823254] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3253.099736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3255.662016] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3260.597419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3260.793575] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3262.979079] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3266.631882] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3269.697530] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3273.661359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3273.871495] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3276.901238] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 3278.011247] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3280.675954] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3289.925766] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3289.934444] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3289.946510] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3289.951400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.957198] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3290.537266] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3290.552693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3290.562730] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3290.567201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.572075] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3291.155601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.737155] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3291.741369] Lustre: Skipped 1 previous similar message [ 3291.751581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3291.756748] Lustre: Skipped 1 previous similar message [ 3291.764746] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3291.772115] Lustre: Skipped 1 previous similar message [ 3291.774635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.783638] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3291.790037] Lustre: Skipped 1 previous similar message [ 3292.270455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.771451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.305770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.930383] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3293.936519] Lustre: Skipped 3 previous similar messages [ 3293.949704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3293.953755] Lustre: Skipped 3 previous similar messages [ 3293.955875] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3293.958482] Lustre: Skipped 3 previous similar messages [ 3293.960793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.974131] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3293.976591] Lustre: Skipped 3 previous similar messages [ 3294.539812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.085236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.689922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.284075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.828799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.439467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.984325] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3297.988168] Lustre: Skipped 6 previous similar messages [ 3297.996020] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3298.008494] Lustre: Skipped 6 previous similar messages [ 3298.018182] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3298.025068] Lustre: Skipped 6 previous similar messages [ 3298.027279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.039078] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3298.043019] Lustre: Skipped 6 previous similar messages [ 3298.591858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.103503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.695087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.196190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.702182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.165414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.690269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.155951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.735302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.304886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.870469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.491519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.002464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.500881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.931589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.489985] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3306.493529] Lustre: Skipped 15 previous similar messages [ 3306.502803] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3306.507791] Lustre: Skipped 15 previous similar messages [ 3306.509784] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3306.512771] Lustre: Skipped 15 previous similar messages [ 3306.515231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.520603] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3306.523848] Lustre: Skipped 15 previous similar messages [ 3306.965302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.477170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.985212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.458524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.951411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.428454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.946467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.441337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.983485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.610442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.099340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.607605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.109588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.559549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.056874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.571539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.050369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.515110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.955970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.437520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.944513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.398019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.888233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.359968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.813425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.261123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.775277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.264381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.691536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.151337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.541506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.974418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.352125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.727553] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3322.730792] Lustre: Skipped 33 previous similar messages [ 3322.740542] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3322.745723] Lustre: Skipped 33 previous similar messages [ 3322.748412] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3322.751461] Lustre: Skipped 33 previous similar messages [ 3322.753217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.757983] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3322.760266] Lustre: Skipped 33 previous similar messages [ 3323.134177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.512306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.882335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.311505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.682857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.117569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.583954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.065374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.540309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.008560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.475673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.988624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.520568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.983091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.419715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.873545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.299681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.747567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.295596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.809881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.262643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.727392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.172977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.676394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.172501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.696577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.191883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.731507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.201131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.687376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.205770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.782366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.241840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.669486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.110962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.568105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.035969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.533798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.972048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.473280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.926134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.391165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.905874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.412261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.918906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.413214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.922501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.405702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.879409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.375490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.896593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.320264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.803129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.296785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.825179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.359946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.832981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.313079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.797912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.269945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.732519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.183758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.684637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.158376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.624551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.079371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.546714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.042912] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3355.048385] Lustre: Skipped 67 previous similar messages [ 3355.060642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3355.071513] Lustre: Skipped 67 previous similar messages [ 3355.078491] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3355.083663] Lustre: Skipped 67 previous similar messages [ 3355.087918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.094623] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3355.100855] Lustre: Skipped 67 previous similar messages [ 3355.566580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.028315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.471104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.932170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.449675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.907943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.354988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.860385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.321211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.883749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.487806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.944364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.392699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.922841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.461428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.964989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.445392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.933094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.428527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.930800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.447640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.903512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.347468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.897561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.407516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.877801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.383433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.926466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.383144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.865302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.340313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.852231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.432814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.980073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.481827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.978357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.445554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.948799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.448732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.943261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.456503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.976033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.462372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.935513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.452926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.882988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.348786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.873184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.334137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.812473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.371774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.859877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.380281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.881485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.406283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.914372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.399246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.915857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.444441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.936373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.404068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.914760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.435649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.922414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.502828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.019539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.525932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.068947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.518072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.050510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.538886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.034804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.625703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.024560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.497467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.038991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.540935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.114868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.579979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.123649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.677427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.135702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.705780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.238473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.730385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.169612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.653867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.151562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.583624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.041583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.524790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.003302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.511971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.097575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.633233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.146417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.659475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.144936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.582671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.055157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.546165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.056152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.572852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.014255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.423695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.886711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.385255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.833834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.336850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.888077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.455293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.973611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.470987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.996766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.531847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.044063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.564749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.113037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.594677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.178500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.750371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.246515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.802492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.339654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.900862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.401803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.863884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.399421] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3419.404868] Lustre: Skipped 127 previous similar messages [ 3419.412589] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3419.418357] Lustre: Skipped 127 previous similar messages [ 3419.420266] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3419.423289] Lustre: Skipped 127 previous similar messages [ 3419.426518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.435546] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3419.440773] Lustre: Skipped 127 previous similar messages [ 3419.964902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.453368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.977725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.567654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.138610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.634781] LustreError: 171719:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3422.639137] LustreError: 171719:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3422.641604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.078772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.598248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.057653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.578318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.168195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.656559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.165558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.721305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.242687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.725189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.161425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.706591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.206624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.731295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.264121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.733287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.238667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.835057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.351123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.838956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.359469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.825211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.332538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.925551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.468305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.923336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.417100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.860203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.341944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.836898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.326113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.794434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.290394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.735738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.096884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.466282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.804945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.139937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.465218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.809449] LustreError: 171719:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3441.813325] LustreError: 171719:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3441.818178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.167019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.578300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.986205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.371614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.759621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.141224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.567907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.990058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.398735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.790521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.211092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.755815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.295726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.772607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.193866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.651243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.147101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.649904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.144890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.613995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.141409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.643027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.110376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.501390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.986608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.437906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.927173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.440750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.013309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.537784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.078422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.616129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.108504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.833263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.412919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.992420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.591455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.204081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.838497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.524526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.171795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.802341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.529340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.187265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.921597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.476703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.105530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.577499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.144865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.769985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.296174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.807202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.312659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.870639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.386729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.933981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.451459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.063407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.647309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.253682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.884822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.453575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.044597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.584726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.092763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.660539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.232614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.794893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.313045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.960298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.496542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.052279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.575296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.121960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.784892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.372049] LustreError: 171719:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3482.380390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.380542] LustreError: 171719:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3482.935602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.477505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.193473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.790487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.402429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.931323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.486248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.040683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.648233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.153336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.668522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.305162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.773551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.296529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.911811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.461074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.010942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.643567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.216385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.841751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.455721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.070525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.597754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.131971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.666920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.190269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.842500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.415510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.953229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.452736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.919081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.414058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.888056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.510438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.169134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.818895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.398608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.937513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.417969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.980136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.479830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.949440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.405619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.918944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.453536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.947454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.493832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.050385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.554062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.012117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.505243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.056241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.693117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.199372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.664187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.175716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.714379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.215163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.736493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.290441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.845256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.387123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.921663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.417927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.897889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.459844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.970162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.485544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.997070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.471503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.886796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.362108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.899489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.421720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.962216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.500947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.014680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.543980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.019451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.501574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.977481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.435899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.919431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.422104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.962477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.480230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.986932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.545118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.074865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.575726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.133612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.627941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.173892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.721770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.274894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.780512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.353797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.902210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.483611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.088253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.726132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.341860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.007623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.531321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.087605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.591711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.179726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.782859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.334988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.904179] LustreError: 171719:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3541.911481] LustreError: 171719:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3541.916197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.393463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.018038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.600635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.241026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.766545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.382885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.965134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.679845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.293742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.950441] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3547.954875] Lustre: Skipped 241 previous similar messages [ 3547.964617] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3547.970632] Lustre: Skipped 241 previous similar messages [ 3547.973820] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3547.976171] Lustre: Skipped 241 previous similar messages [ 3547.978690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.983457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3547.986409] Lustre: Skipped 241 previous similar messages [ 3548.635210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.469870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.031781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.565863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.215276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.709414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.264591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.888885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.405874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.920071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3554.428438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3554.966477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3555.371442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3555.806830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.234915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.699118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3557.231431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3557.762240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.239540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.771973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3559.390595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3559.927418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3560.433486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3561.010600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3561.637622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3562.183778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3562.771873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3563.415426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.001785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.828100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3565.489640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.071178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.610590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3567.176367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3567.689792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3568.173904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3568.713137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3569.306636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3569.799155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.305689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.887800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3571.409646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3571.873326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.363166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.857640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3573.279937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3573.742336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3574.203051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3574.635807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3575.150533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3575.648717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.186307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.929227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3577.529398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3578.147473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3578.562273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.013508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.506805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.949811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3580.389880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3580.842587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3581.222205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3581.620296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.028428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.471744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.962690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.416775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.962199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3584.409758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3584.863449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3585.329408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3585.791348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3586.210088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3586.612833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.037933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.438099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.806288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3588.271903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3588.700094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.104396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.551362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.027505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.525720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3591.080713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3591.570746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3592.041621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3592.552855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.094391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.506703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.923032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.417389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.887665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3595.373674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3595.791416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3596.239768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3596.819528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3597.448411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3597.973172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3598.437136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3598.936256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.447795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.908037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3600.362741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3600.926309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3601.424833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3601.874847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.337339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.803586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.277064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.693195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.090304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.491615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.891869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.246122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.675670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.089853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.541765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.950401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.383466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.875303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.348166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.836270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.313629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.727073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.208763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.700415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.175105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.707581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.155598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.600507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.078315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.609015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.027141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.468440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.929812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.419954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.914270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.443620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.029261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.494732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.984218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.554932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.158334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.610731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.032496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.418940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.798441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.167156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.599389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.045131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.536586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.063446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.545629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.018747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.382227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.759892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.146935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.593421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.010609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.436121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.779653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.143162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.580355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.045433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.472265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.972122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.479179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.931468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.404911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.829832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.264128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.697444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.133665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.515797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.972672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.382849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.765119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.154694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.573976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.982243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.460334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.871869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.290847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.751762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.239509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.708553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.223731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.669083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.068913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.512164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.913340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.309581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.701393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.152709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.542690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.914752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.356527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.804375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.205539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.627317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.033168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.388960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.765520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.163822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.617548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.068779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.549537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.097630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.597727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.091544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.563240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.021774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.467826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.049521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.494832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.998442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.557204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3651.985772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.448495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.925340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.397795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.904633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.416880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.863822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.290422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.777338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.245365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.761542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.250711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.757893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.236898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.817218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.293675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.781472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.266563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.752405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.219889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.785360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.313549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.776954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.228863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.627132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.107666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.504695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.940701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.436952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.908430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.400459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.818959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.272719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.718078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.150788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.602760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.102275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.579381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.018911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.435496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.826142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.236080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.629771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.023944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.424600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.761276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.100108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.428879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.792614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.141716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.511065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.911423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.297204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.735782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.190422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.557296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.919319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.280199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.627545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.971682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.377760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.771715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.169645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.655447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.081682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.502676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.043875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.478190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.917977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.395816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.848930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.383826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.988349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.567452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.109864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.548184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.026672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.525207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.937432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.382015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.840014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.216441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.654924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.184761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.591882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.093841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.607294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.005553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.461172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.906987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.319461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.777284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.246409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.823748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.302659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.734552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.195680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.671055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.096154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.493607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.856224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.248823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.681215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.122411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.521703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.932934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.312318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.707346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.186655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.626935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.056372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.496713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.902680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.332413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.853824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.451768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.939991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.367109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.849451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.327153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.833654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.345206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.810796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.332858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.775731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.316134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.874465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.456639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.057141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.542452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.106073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.807713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3712.347848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3712.919159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.456586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.983738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3714.494356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3715.010271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3715.526780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.007294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.483163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.022653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.457310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.033778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.509428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.062886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.505848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.927947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3720.404983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3720.849472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.380763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.898109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.353538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.811486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.211577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.651353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.014384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.452707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.883708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3725.368124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3725.853641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.259076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.658531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.174521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.632781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.119535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.572343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.020835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.575700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.050443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.615153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3731.116197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3731.611379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.101238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.645545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.171025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.691975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.228506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.770643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.312753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.857927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.436182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.969235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3737.451311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.021886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.510310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.963321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3739.490333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.016614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.580806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.044200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.544508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.015505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.536974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.026847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.577866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.041811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.461266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.935376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.433970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.865641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.388380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.924427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.441721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.929986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.367090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.758937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.120869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.494161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.936823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.360684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.750223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.159559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.616719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.034149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.511870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.960812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.375705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.813033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.312111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.817821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.287586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.738209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.194703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.712771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.226489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.712316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.211623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.650480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.121583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.630501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.060426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.612799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.184777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.712560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.213383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.703995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.224553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.691794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.194858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.750205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.237872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.810233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.398424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.969528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.454513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.951151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.426732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.879607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.372641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.912343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.410125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.888511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.372646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.901701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.374732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.880060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.368916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.834494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.333847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.838683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.348256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.887233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.359584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.929751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.426326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.923969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.481519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.989311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.495502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.061931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.512628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.011624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.488480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.980614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.505164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.991850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.506394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.037420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.546679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.206406] LustreError: 173519:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4029.210194] LustreError: 171719:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4175.328329] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4175.331357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4175.336147] Lustre: Skipped 497 previous similar messages [ 4175.340122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4176.354963] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4176.358639] Lustre: Skipped 1 previous similar message [ 4179.584836] Lustre: server umount lustre-OST0000 complete [ 4181.473200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4181.475773] Lustre: Skipped 1 previous similar message [ 4186.594269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4187.302915] Lustre: server umount lustre-MDT0000 complete [ 4188.934581] LustreError: 225742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097020 with bad export cookie 2609177480258723518 [ 4188.939141] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4188.939766] LustreError: 225742:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4189.096365] Lustre: server umount lustre-MDT0001 complete [ 4193.510472] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4194.903926] Key type lgssc unregistered [ 4195.054750] LNet: 226781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4195.059704] LNetError: 226781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4195.067499] LNet: Removed LNI 192.168.202.102@tcp [ 4195.452671] Key type .llcrypt unregistered [ 4195.454452] Key type ._llcrypt unregistered [ 4206.320485] Key type ._llcrypt registered [ 4206.322067] Key type .llcrypt registered [ 4206.369248] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4206.782808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4206.812032] alg: No test for adler32 (adler32-zlib) [ 4207.687074] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 4207.788164] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 4209.383675] Key type lgssc registered [ 4209.879252] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4213.322194] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4213.331963] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4214.557476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4216.334790] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4220.271618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4220.472918] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4222.237814] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4225.074798] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4227.342772] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4230.061164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4230.211364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4232.479832] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4235.510806] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4239.462195] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 4243.807562] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4244.599367] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4244.604761] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4244.609184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4244.613378] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4244.613391] LustreError: 228195:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4246.946688] Lustre: lustre-OST0000: Not available for connect from 192.168.202.2@tcp (stopping) [ 4247.009234] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4247.012852] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4250.088662] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4250.090387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4250.091617] Lustre: Skipped 1 previous similar message [ 4251.488163] Lustre: server umount lustre-OST0000 complete [ 4264.695170] LustreError: 228195:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4280.774991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4280.967320] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4283.770942] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4286.925816] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4287.588408] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 4289.871343] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92b084309000.ost_server_uuid 50 [ 4290.212714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4290.218490] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4290.581113] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92b084309000.ost_server_uuid in FULL state after 0 sec [ 4295.653578] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4295.659231] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4295.661467] Lustre: Skipped 1 previous similar message [ 4298.138266] Lustre: server umount lustre-OST0000 complete [ 4302.305456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4302.310954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4302.317060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4306.030641] Lustre: server umount lustre-MDT0000 complete [ 4307.753317] LustreError: 228154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097139 with bad export cookie 122966277011259983 [ 4307.758907] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4307.759350] LustreError: 228154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4307.907176] Lustre: server umount lustre-MDT0001 complete [ 4312.732566] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4314.166193] Key type lgssc unregistered [ 4314.304404] LNet: 233385:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4314.307378] LNetError: 233385:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4314.316439] LNet: Removed LNI 192.168.202.102@tcp [ 4314.747360] Key type .llcrypt unregistered [ 4314.749181] Key type ._llcrypt unregistered [ 4329.537983] Key type ._llcrypt registered [ 4329.539070] Key type .llcrypt registered [ 4329.578728] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4329.987864] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4330.010553] alg: No test for adler32 (adler32-zlib) [ 4330.931810] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 4331.067313] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 4332.671527] Key type lgssc registered [ 4333.235353] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4337.419451] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4337.427848] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4338.623354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4340.370528] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4344.074456] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4344.280484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4346.129091] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4347.412200] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4350.232698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4350.407645] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4352.898903] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4356.753837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4356.848461] Lustre: lustre-OST0001: new disk, initializing [ 4356.851277] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4356.855389] Lustre: 237396:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4356.920305] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4357.471085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4357.475524] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4357.542627] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4359.617728] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4363.695478] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 4366.867706] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4373.016536] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4384.231352] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4384.233034] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4384.235074] Lustre: Skipped 2 previous similar messages [ 4384.241637] Lustre: Skipped 1 previous similar message [ 4387.251652] Lustre: server umount lustre-MDT0000 complete [ 4388.829248] LustreError: 238169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097220 with bad export cookie 17163112820392075947 [ 4388.832151] LustreError: 235827:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 4388.832190] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4388.833201] LustreError: 238169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4388.976736] Lustre: server umount lustre-MDT0001 complete [ 4400.779983] Lustre: server umount lustre-OST0000 complete [ 4411.450256] Lustre: server umount lustre-OST0001 complete [ 4420.399897] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4425.203311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4425.407949] LustreError: 240511:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4425.419178] LustreError: 240511:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4425.443696] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4427.430656] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4430.817670] LustreError: 240512:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4431.319103] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4433.207396] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4434.539486] Lustre: 241653:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4437.169442] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4437.301858] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4437.304462] Lustre: Skipped 1 previous similar message [ 4439.465270] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4441.381163] LustreError: 242621:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4441.384195] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 4442.709840] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4445.125219] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4448.236738] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 4451.307682] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 4459.193324] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4460.624963] Lustre: 243544:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4466.658953] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4466.660600] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4466.663185] Lustre: Skipped 3 previous similar messages [ 4466.664812] Lustre: Skipped 1 previous similar message [ 4470.693783] Lustre: server umount mylustre-MDT0000 complete [ 4471.778201] LustreError: 240511:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4471.784438] LustreError: 240511:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4472.117974] LustreError: 240487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097303 with bad export cookie 17163112820392078775 [ 4472.118574] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4472.122304] LustreError: 240487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4472.261106] Lustre: server umount mylustre-MDT0001 complete [ 4484.105587] Lustre: server umount mylustre-OST0000 complete [ 4495.789336] Lustre: server umount mylustre-OST0001 complete [ 4503.188256] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4506.668157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4506.842958] LustreError: 245851:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4506.871507] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4506.873529] Lustre: Skipped 1 previous similar message [ 4508.295994] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4511.298357] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4512.795487] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4513.838100] Lustre: 246994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4516.202608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4517.347111] LustreError: 247349:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4517.354036] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 4518.263590] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4521.277664] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4522.413106] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 4522.414977] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 4523.360312] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4526.455434] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4527.673697] Lustre: 248863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4532.192212] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4532.195510] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.201266] Lustre: Skipped 1 previous similar message [ 4532.204269] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4535.911937] Lustre: server umount tfs-MDT0000 complete [ 4537.364623] LustreError: 245828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097369 with bad export cookie 17163112820392080952 [ 4537.367925] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4537.369167] LustreError: 245828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4537.518697] Lustre: server umount tfs-MDT0001 complete [ 4549.255430] Lustre: server umount tfs-OST0000 complete [ 4560.862231] Lustre: server umount tfs-OST0001 complete [ 4568.254238] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4571.877180] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.025461] LustreError: 251109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4572.043724] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4572.046225] Lustre: Skipped 3 previous similar messages [ 4573.527643] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4576.640049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4578.209145] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4579.311013] Lustre: 252250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4581.660641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4582.830219] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 4584.025249] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4586.922757] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 4587.306597] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4589.588945] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4592.614970] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 4592.945368] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4594.360967] Lustre: 254120:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4596.806671] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 02:37:08 (1784097428) [ 4612.578160] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4612.582804] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4612.588479] Lustre: Skipped 3 previous similar messages [ 4612.590949] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4612.593371] Lustre: Skipped 3 previous similar messages [ 4614.879958] Lustre: server umount lustre-MDT0000 complete [ 4616.335333] LustreError: 251084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097447 with bad export cookie 17163112820392082863 [ 4616.340532] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4616.342609] LustreError: 251084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4618.208044] LustreError: 253934:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4618.210545] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4618.211396] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4618.211400] Lustre: Skipped 4 previous similar messages [ 4618.214308] LustreError: 253934:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4618.218026] Lustre: Skipped 4 previous similar messages [ 4622.515512] Lustre: server umount lustre-MDT0001 complete [ 4638.175117] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4638.228247] Lustre: server umount lustre-OST0000 complete [ 4645.656430] Lustre: server umount lustre-OST0001 complete [ 4648.148800] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 4651.989091] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4665.838940] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4665.838940] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4665.838940] [ 4665.846388] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4665.936953] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4665.952930] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4665.988295] Lustre: lustre-MDT0000: new disk, initializing [ 4666.016333] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4666.019624] Lustre: Skipped 3 previous similar messages [ 4666.027405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4667.533017] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4672.310586] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4672.310586] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4672.310586] [ 4672.320650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4672.369125] Lustre: 257301:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4672.401109] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4672.404336] Lustre: Skipped 1 previous similar message [ 4672.445105] Lustre: lustre-MDT0001: new disk, initializing [ 4672.505065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4672.508927] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4674.005884] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4678.546301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4678.681438] Lustre: lustre-OST0000: new disk, initializing [ 4678.683742] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4678.687436] Lustre: 258883:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4680.386653] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4680.390816] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4680.438445] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4680.882844] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4685.421442] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4685.509402] Lustre: lustre-OST0001: new disk, initializing [ 4685.511687] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4685.515191] Lustre: 259753:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4686.793571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4686.797250] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4686.830607] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4687.779812] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4692.674175] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 02:38:43 (1784097523) [ 4697.057420] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 02:38:48 (1784097528) [ 4711.396105] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4711.396495] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4711.399439] LustreError: Skipped 1 previous similar message [ 4711.400202] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4711.403762] Lustre: Skipped 1 previous similar message [ 4711.405593] Lustre: Skipped 3 previous similar messages [ 4714.626854] Lustre: server umount lustre-OST0000 complete [ 4717.538070] LustreError: 258867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4717.543287] LustreError: 258867:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4727.776630] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4727.780308] Lustre: Skipped 11 previous similar messages [ 4730.335176] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4730.424280] Lustre: server umount lustre-MDT0000 complete [ 4731.839432] LustreError: 257292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097563 with bad export cookie 17163112820392084641 [ 4731.843852] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4731.844886] LustreError: 257292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4732.896894] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4732.900886] Lustre: Skipped 3 previous similar messages [ 4746.207821] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4746.308576] Lustre: server umount lustre-MDT0001 complete [ 4750.115644] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4760.221726] Lustre: server umount lustre-OST0001 complete [ 4762.319350] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 4768.607906] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4781.418294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4781.518118] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4781.530510] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4781.569259] Lustre: lustre-MDT0000: new disk, initializing [ 4781.597097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4783.171703] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4787.593819] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4787.640180] Lustre: 264280:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4787.753136] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4789.178243] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4793.164447] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4795.194728] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4797.604808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4797.723424] Lustre: lustre-OST0000: new disk, initializing [ 4797.724895] Lustre: Skipped 1 previous similar message [ 4797.726818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4797.729043] Lustre: Skipped 2 previous similar messages [ 4797.731469] Lustre: 266246:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4797.764763] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4797.767494] Lustre: Skipped 5 previous similar messages [ 4799.227442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4799.229997] Lustre: Skipped 1 previous similar message [ 4799.232415] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4799.276248] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4799.838510] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4803.622760] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4809.699251] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4809.699383] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4809.700390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4809.700394] Lustre: Skipped 3 previous similar messages [ 4809.708752] Lustre: Skipped 1 previous similar message [ 4812.114051] Lustre: server umount lustre-OST0000 complete [ 4819.549116] Lustre: server umount lustre-MDT0000 complete [ 4820.854942] LustreError: 267254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097652 with bad export cookie 17163112820392086111 [ 4820.857552] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4820.858209] LustreError: 267254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4820.988106] Lustre: server umount lustre-MDT0001 complete [ 4824.673822] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4825.695107] Key type lgssc unregistered [ 4825.825393] LNet: 268004:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4825.828288] LNetError: 268004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4825.836268] LNet: Removed LNI 192.168.202.102@tcp [ 4826.139143] Key type .llcrypt unregistered [ 4826.140368] Key type ._llcrypt unregistered [ 4835.580610] Key type ._llcrypt registered [ 4835.581717] Key type .llcrypt registered [ 4835.618096] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4836.027145] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4836.041525] alg: No test for adler32 (adler32-zlib) [ 4836.898562] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 4836.995727] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 4838.575192] Key type lgssc registered [ 4838.933513] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4841.504818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4841.510862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4842.660312] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4843.996818] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4846.818340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4846.947169] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4848.271593] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4850.721298] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4852.669032] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4855.270349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4855.423663] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4857.771100] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4860.462875] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4860.834953] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4861.606211] Lustre: server umount lustre-OST0000 complete [ 4867.040711] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4867.041314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4867.043569] Lustre: Skipped 1 previous similar message [ 4869.082148] Lustre: server umount lustre-MDT0000 complete [ 4870.423822] LustreError: 269425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097702 with bad export cookie 10664373707776740241 [ 4870.426845] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4870.427819] LustreError: 269425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4870.433207] LustreError: 270174:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0001: can't precreate: rc = -5 [ 4870.436906] LustreError: 270174:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0001: cannot precreate objects: rc = -5 [ 4880.931402] Lustre: server umount lustre-MDT0001 complete [ 4884.860536] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4885.896075] Key type lgssc unregistered [ 4886.023379] LNet: 273029:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4886.026288] LNetError: 273029:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4886.035310] LNet: Removed LNI 192.168.202.102@tcp [ 4886.382219] Key type .llcrypt unregistered [ 4886.383644] Key type ._llcrypt unregistered [ 4892.926676] Key type ._llcrypt registered [ 4892.927588] Key type .llcrypt registered [ 4892.966219] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4893.285740] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4893.293090] alg: No test for adler32 (adler32-zlib) [ 4894.147222] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 4894.238680] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 4895.823155] Key type lgssc registered [ 4896.175476] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4899.185068] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4899.191482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4914.655430] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4919.775280] LustreError: 274199:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.102@tcp: failed processing log, type 4: rc = -110 [ 4946.399182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4948.368922] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 4950.945194] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4951.696325] Lustre: server umount lustre-OST0000 complete [ 4957.244428] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 4958.352258] Key type lgssc unregistered [ 4958.498633] LNet: 275826:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4958.502053] LNetError: 275826:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4958.510532] LNet: Removed LNI 192.168.202.102@tcp [ 4958.874381] Key type .llcrypt unregistered [ 4958.875539] Key type ._llcrypt unregistered [ 4973.817401] Key type ._llcrypt registered [ 4973.818807] Key type .llcrypt registered [ 4973.868253] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 4974.414250] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4974.425799] alg: No test for adler32 (adler32-zlib) [ 4975.293569] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 4975.409212] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 4976.999161] Key type lgssc registered [ 4977.401347] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4982.145916] loop0: detected capacity change from 0 to 204800000 [ 4982.260778] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4982.266592] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4982.398021] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4982.763960] loop0: detected capacity change from 0 to 204800000 [ 4982.819642] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4982.823467] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4982.953311] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4983.336598] loop0: detected capacity change from 0 to 204800000 [ 4983.404381] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4983.408496] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4983.454470] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4983.959284] loop0: detected capacity change from 0 to 204800000 [ 4984.044232] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4984.488215] loop0: detected capacity change from 0 to 204800000 [ 4984.500831] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4985.970249] loop0: detected capacity change from 0 to 204800000 [ 4985.985774] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4987.475422] loop0: detected capacity change from 0 to 204800000 [ 4987.488882] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4989.105861] loop0: detected capacity change from 0 to 204800000 [ 4989.125803] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4990.550843] loop0: detected capacity change from 0 to 204800000 [ 4990.592810] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4990.604237] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4990.639879] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4990.643654] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4990.647620] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4990.652705] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4990.656890] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4990.661049] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4990.665454] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4990.669469] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4990.673615] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4990.677853] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4990.681803] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4990.685926] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4990.690206] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4990.694432] Lustre: 279216:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4993.470914] Lustre: server umount MGS complete [ 4993.847977] loop0: detected capacity change from 0 to 204800000 [ 4993.879458] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4994.138740] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4994.159070] Lustre: lustre-MDT0000: Aborting client recovery [ 4994.550651] loop1: detected capacity change from 0 to 204800000 [ 4994.583667] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4994.627585] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4994.632318] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4994.637191] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4994.643867] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4994.651191] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4994.658233] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4994.661143] Lustre: 279646:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4994.668251] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4994.741466] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4994.771318] Lustre: lustre-MDT0001: Aborting client recovery [ 4995.099925] loop2: detected capacity change from 0 to 204800000 [ 4995.135824] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4995.182831] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4995.189162] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4995.195703] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4995.202434] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4995.210849] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4995.217539] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4995.223635] Lustre: 279729:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4995.282796] Lustre: Found index 0 for lustre-OST0000, updating log [ 4995.294678] Lustre: 279747:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4995.316639] Lustre: lustre-OST0000: Aborting recovery [ 4995.631803] loop3: detected capacity change from 0 to 204800000 [ 4995.670949] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4995.701243] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4995.705567] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4995.709079] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4995.714114] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4995.718591] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4995.722696] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4995.727807] Lustre: 279813:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4995.759697] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4995.762205] Lustre: Skipped 1 previous similar message [ 5001.188734] LustreError: 279590:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 5004.256831] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5004.257208] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 5004.269150] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 5004.269973] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 5005.281467] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 5005.282929] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 5006.208792] LustreError: 279557:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 247us: rc = -2 [ 5020.639804] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5020.643898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5020.647651] Lustre: Skipped 1 previous similar message [ 5023.649392] Lustre: server umount lustre-MDT0000 complete [ 5024.010333] LustreError: 279542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784097855 with bad export cookie 15637313593085134158 [ 5024.012518] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5024.016838] LustreError: 279542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5025.759721] LustreError: 279555:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5025.759864] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5025.760191] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5025.760195] Lustre: Skipped 3 previous similar messages [ 5025.768538] LustreError: 279555:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5025.783888] Lustre: Skipped 3 previous similar messages [ 5029.791621] LustreError: 280105:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5029.791628] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5029.791637] Lustre: Skipped 1 previous similar message [ 5029.796731] LustreError: 280105:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5030.202147] Lustre: server umount lustre-MDT0001 complete [ 5036.683711] Lustre: server umount lustre-OST0000 complete [ 5051.359140] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5051.416295] Lustre: server umount lustre-OST0001 complete [ 5054.254060] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 02:44:45 (1784097885) [ 5064.085643] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 5067.380407] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5080.563900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5080.660775] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5080.673060] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5080.715602] Lustre: lustre-MDT0000: new disk, initializing [ 5080.747373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5080.756920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5082.147739] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5086.308201] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5086.350904] Lustre: 283875:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5086.373460] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5086.375303] Lustre: Skipped 1 previous similar message [ 5086.412370] Lustre: lustre-MDT0001: new disk, initializing [ 5086.446482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5086.459443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5086.462643] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5087.793636] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5091.731805] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5093.695082] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5095.949347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5096.053931] Lustre: lustre-OST0000: new disk, initializing [ 5096.055983] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5096.058540] Lustre: 285844:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5096.061042] Lustre: 285844:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 5096.089200] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5097.944401] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5101.525827] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5104.120774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5104.124636] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5104.157735] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5109.341590] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5111.836736] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5117.692045] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5122.175906] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 5160.418693] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5160.419109] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5160.422897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5160.426049] Lustre: Skipped 1 previous similar message [ 5165.537287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5165.539247] Lustre: Skipped 1 previous similar message [ 5172.191168] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5172.249558] Lustre: server umount lustre-OST0000 complete [ 5173.727892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5173.730369] LustreError: Skipped 1 previous similar message [ 5173.731587] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5173.735207] Lustre: Skipped 1 previous similar message [ 5173.736837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5173.738704] Lustre: Skipped 2 previous similar messages [ 5179.619466] Lustre: server umount lustre-MDT0000 complete [ 5180.896141] LustreError: 283881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5180.901013] LustreError: 283881:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5180.919226] LustreError: 284670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098012 with bad export cookie 15637313593085136867 [ 5180.921874] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5180.923014] LustreError: 284670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5181.033569] Lustre: server umount lustre-MDT0001 complete [ 5184.006302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5185.431220] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5187.081468] Lustre: server umount MGS complete [ 5190.829624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5190.992563] LustreError: 289658:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5190.999536] LustreError: 289671:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5191.021537] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5192.364368] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5195.154506] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5196.717832] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5199.152512] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5201.087858] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5203.506732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5205.978316] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5208.514130] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5240.799891] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5240.803499] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5240.808677] Lustre: Skipped 1 previous similar message [ 5240.811196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5240.813721] Lustre: Skipped 2 previous similar messages [ 5242.852564] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5245.068763] Lustre: server umount lustre-OST0000 complete [ 5252.443689] Lustre: server umount lustre-MDT0000 complete [ 5253.087977] LustreError: 292148:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5253.093954] LustreError: 292148:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5253.720426] LustreError: 289639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098085 with bad export cookie 15637313593085138680 [ 5253.722960] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5253.724958] LustreError: 289639:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5253.840107] Lustre: server umount lustre-MDT0001 complete [ 5257.579294] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 5258.620371] Key type lgssc unregistered [ 5258.752452] LNet: 293620:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5258.756366] LNetError: 293620:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5258.765419] LNet: Removed LNI 192.168.202.102@tcp [ 5259.078816] Key type .llcrypt unregistered [ 5259.079843] Key type ._llcrypt unregistered [ 5270.219762] Key type ._llcrypt registered [ 5270.221418] Key type .llcrypt registered [ 5270.257885] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 5276.134607] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5276.535382] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5276.566139] alg: No test for adler32 (adler32-zlib) [ 5277.400959] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 5277.489649] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 5279.063172] Key type lgssc registered [ 5279.387218] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5290.857643] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5290.863535] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5291.956678] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5291.967714] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5292.010658] Lustre: lustre-MDT0000: new disk, initializing [ 5292.032084] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5292.038953] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5293.402469] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5297.640755] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5297.692390] Lustre: 297547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5297.724440] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5297.727771] Lustre: Skipped 1 previous similar message [ 5297.774420] Lustre: lustre-MDT0001: new disk, initializing [ 5297.815159] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5297.829186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5297.833661] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5299.277139] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5303.120378] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5305.144849] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5307.439038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5307.572028] Lustre: lustre-OST0000: new disk, initializing [ 5307.573622] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5307.575973] Lustre: 299515:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5307.610292] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5309.582115] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5313.251588] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5317.110842] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5317.114048] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5317.144058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5321.810060] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 5324.392837] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 5334.072276] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 5383.650027] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5383.650563] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5383.652035] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5383.652280] LustreError: Skipped 1 previous similar message [ 5383.658701] Lustre: Skipped 1 previous similar message [ 5388.769764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5388.771783] Lustre: Skipped 2 previous similar messages [ 5392.863173] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5392.915441] Lustre: server umount lustre-OST0000 complete [ 5395.423899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5395.427354] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5395.434246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5399.008257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5399.008835] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5400.285652] Lustre: server umount lustre-MDT0000 complete [ 5401.548293] LustreError: 300369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098233 with bad export cookie 747514584284881047 [ 5401.551937] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5401.552348] LustreError: 300369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5401.681870] Lustre: server umount lustre-MDT0001 complete [ 5404.510899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5405.957309] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5407.646102] Lustre: server umount MGS complete [ 5411.443912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5411.613099] LustreError: 303504:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 5411.636628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5413.003668] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5415.944114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5416.069749] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5417.470073] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5419.846985] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5421.711397] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5424.055786] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5424.184379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5426.199912] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5428.780403] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5456.863881] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5456.867609] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5456.872571] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5456.874398] Lustre: Skipped 1 previous similar message [ 5461.394447] Lustre: server umount lustre-OST0000 complete [ 5463.008625] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5463.012719] Lustre: Skipped 2 previous similar messages [ 5468.128153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5468.130235] Lustre: Skipped 5 previous similar messages [ 5477.343126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5477.416459] Lustre: server umount lustre-MDT0000 complete [ 5478.367764] LustreError: 303491:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5478.372458] LustreError: 303491:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5478.671094] LustreError: 306240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098310 with bad export cookie 747514584284882860 [ 5478.674720] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5478.677573] LustreError: 306240:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5478.799825] Lustre: server umount lustre-MDT0001 complete [ 5482.342426] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 5483.407942] Key type lgssc unregistered [ 5483.539376] LNet: 307457:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5483.541856] LNetError: 307457:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5483.549247] LNet: Removed LNI 192.168.202.102@tcp [ 5483.867288] Key type .llcrypt unregistered [ 5483.868163] Key type ._llcrypt unregistered [ 5490.062501] Key type ._llcrypt registered [ 5490.063856] Key type .llcrypt registered [ 5490.105725] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 5495.776941] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5496.075778] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5496.135895] alg: No test for adler32 (adler32-zlib) [ 5496.976278] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 5497.065869] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 5498.647198] Key type lgssc registered [ 5499.018436] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5506.583048] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5511.361895] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5511.367952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5512.464837] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5512.476022] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5512.518707] Lustre: lustre-MDT0000: new disk, initializing [ 5512.543726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5512.549520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5513.882813] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5518.067025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5518.177630] Lustre: lustre-OST0000: new disk, initializing [ 5518.179312] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5518.180930] Lustre: Skipped 1 previous similar message [ 5518.182823] Lustre: 311477:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5518.208569] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5520.036111] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5521.977353] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 5521.980796] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 5521.989384] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5701.088247] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5701.094664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5705.897282] Lustre: server umount lustre-MDT0000 complete [ 5707.174130] LustreError: 311475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098538 with bad export cookie 18314658295962856728 [ 5707.176266] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5707.178084] LustreError: 311475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5707.239758] Lustre: server umount lustre-OST0000 complete [ 5715.374388] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 5718.803398] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5730.815561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5730.918141] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5730.930436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5730.969383] Lustre: lustre-MDT0000: new disk, initializing [ 5730.990649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5730.995951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5732.235792] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5736.317019] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5736.363101] Lustre: 315706:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5736.385780] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5736.387963] Lustre: Skipped 1 previous similar message [ 5736.425954] Lustre: lustre-MDT0001: new disk, initializing [ 5736.459422] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5736.472291] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5736.476114] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5737.853170] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5741.571234] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5743.350688] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5745.527180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5745.622135] Lustre: lustre-OST0000: new disk, initializing [ 5745.623548] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5745.625967] Lustre: 317672:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5745.650269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5746.007442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5746.011869] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5746.041338] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5747.586531] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5751.240723] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5752.584742] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5752.657985] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5753.973304] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5754.041146] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5756.384952] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5756.385116] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5756.388042] LustreError: Skipped 1 previous similar message [ 5756.393149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5756.395861] Lustre: Skipped 1 previous similar message [ 5760.782174] Lustre: server umount lustre-OST0000 complete [ 5766.624573] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5766.625274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5766.628739] Lustre: Skipped 2 previous similar messages [ 5771.744096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5771.746502] Lustre: Skipped 2 previous similar messages [ 5776.351135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5776.416279] Lustre: server umount lustre-MDT0000 complete [ 5777.712423] LustreError: 315697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098609 with bad export cookie 18314658295965797505 [ 5777.715685] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5777.716821] LustreError: 315697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5777.849159] Lustre: server umount lustre-MDT0001 complete [ 5779.101297] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5779.747898] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 02:56:50 (1784098610) [ 5782.110048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5782.274708] LustreError: 319986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5782.282088] LustreError: 319986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5782.306330] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5783.770887] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5786.741561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5788.424730] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5791.093540] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5793.152855] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5795.585981] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5797.773970] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5800.442695] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5802.884539] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5802.944264] Lustre: lustre-OST0001: new disk, initializing [ 5802.946099] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5802.948809] Lustre: 322820:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5802.980766] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5802.983475] Lustre: Skipped 2 previous similar messages [ 5805.046162] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5805.572280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5805.575564] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5805.589748] LustreError: 323252:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5806.630114] LustreError: 323252:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5806.633448] LustreError: 323252:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5807.650757] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5807.654695] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5808.863171] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5808.933908] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5809.699519] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5809.703073] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5810.728585] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92b0a12f8800.ost_server_uuid 50 [ 5811.214921] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92b0a12f8800.ost_server_uuid in FULL state after 0 sec [ 5813.015114] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92b0a12f8800.ost_server_uuid 50 [ 5813.519404] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff92b0a12f8800.ost_server_uuid in FULL state after 0 sec [ 5813.794099] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5813.797179] LustreError: 323262:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5814.825547] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5827.043679] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5827.045217] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5827.048236] Lustre: Skipped 1 previous similar message [ 5831.226088] Lustre: server umount lustre-OST0001 complete [ 5832.164684] LustreError: 321804:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5832.170164] LustreError: 321804:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5837.281516] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5837.282742] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5837.283144] LustreError: 321804:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5837.285127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5837.287769] Lustre: Skipped 1 previous similar message [ 5837.291820] Lustre: Skipped 2 previous similar messages [ 5838.874516] Lustre: server umount lustre-OST0000 complete [ 5846.304969] Lustre: server umount lustre-MDT0000 complete [ 5847.519831] LustreError: 320005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5847.523706] LustreError: 320005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5847.638456] LustreError: 322819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098679 with bad export cookie 18314658295965798373 [ 5847.641258] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5847.642133] LustreError: 322819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5847.761241] Lustre: server umount lustre-MDT0001 complete [ 5851.524982] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 5852.527199] Key type lgssc unregistered [ 5852.653387] LNet: 325504:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5852.657199] LNetError: 325504:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5852.663483] LNet: Removed LNI 192.168.202.102@tcp [ 5852.968286] Key type .llcrypt unregistered [ 5852.969607] Key type ._llcrypt unregistered [ 5861.980230] Key type ._llcrypt registered [ 5861.981775] Key type .llcrypt registered [ 5862.015052] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 5862.342797] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5862.378513] alg: No test for adler32 (adler32-zlib) [ 5863.225467] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 5863.314173] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 5864.895148] Key type lgssc registered [ 5865.192652] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5867.694858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5867.701953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5868.852887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5870.285402] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5872.759102] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5874.999375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5875.116553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5876.429582] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5879.662961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5879.776652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5881.715760] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5884.250960] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5884.899563] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5884.906878] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5886.590352] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5886.665304] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5888.570273] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 5890.978160] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5894.123674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5894.129082] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5898.113392] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5898.189504] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5973.983851] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5973.985806] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5973.989718] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5976.035803] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5976.036479] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5976.037521] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5979.318762] Lustre: server umount lustre-OST0001 complete [ 5981.152931] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5981.155021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5981.155474] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5981.157067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5981.157072] Lustre: Skipped 1 previous similar message [ 5981.158234] Lustre: Skipped 1 previous similar message [ 5981.162547] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5986.273022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5986.273192] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5986.275435] Lustre: Skipped 2 previous similar messages [ 5986.282438] LustreError: 329302:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5987.021233] Lustre: server umount lustre-OST0000 complete [ 5988.307862] Lustre: server umount lustre-MDT0000 complete [ 5989.513286] LustreError: 326882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098821 with bad export cookie 7765695384527229962 [ 5989.516985] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5989.640305] Lustre: server umount lustre-MDT0001 complete [ 5993.097260] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 5994.152809] Key type lgssc unregistered [ 5994.284663] LNet: 331831:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5994.287121] LNetError: 331831:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5994.293374] LNet: Removed LNI 192.168.202.102@tcp [ 5994.613159] Key type .llcrypt unregistered [ 5994.614162] Key type ._llcrypt unregistered [ 6011.941611] Key type ._llcrypt registered [ 6011.942601] Key type .llcrypt registered [ 6011.983705] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 6012.332994] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6012.395712] alg: No test for adler32 (adler32-zlib) [ 6013.253813] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 6013.347146] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 6014.927148] Key type lgssc registered [ 6015.274271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6018.301455] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6018.306893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6019.472864] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6020.925282] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6023.857030] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6024.003954] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6025.400783] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6026.439828] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6028.792395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6028.934375] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6029.988640] LustreError: 335583:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6029.994582] LustreError: 335583:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6029.995898] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 6030.991945] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6033.904556] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6033.995544] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6035.061865] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 6036.020587] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6040.283048] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6042.978720] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6049.760111] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6049.763629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6049.769406] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6050.784845] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6050.786503] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6050.792553] Lustre: Skipped 2 previous similar messages [ 6052.443790] Lustre: server umount lustre-MDT0000 complete [ 6053.737413] LustreError: 335232:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098885 with bad export cookie 5212123293102422488 [ 6053.741594] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6053.742497] LustreError: 335232:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6053.869987] Lustre: server umount lustre-MDT0001 complete [ 6065.091458] Lustre: server umount lustre-OST0000 complete [ 6075.559863] Lustre: server umount lustre-OST0001 complete [ 6081.033317] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 6084.737427] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6084.889843] LustreError: 339366:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6084.906148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6086.307342] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6089.094490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6090.613118] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6091.593621] Lustre: 340507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6093.857306] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6094.000378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6094.002257] Lustre: Skipped 1 previous similar message [ 6095.010979] LustreError: 340862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6095.015210] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 6095.959764] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6098.915178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6100.074213] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 6100.918444] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6104.042659] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6105.247453] Lustre: 342377:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6109.664122] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6109.666612] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6109.670480] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6109.671968] Lustre: Skipped 3 previous similar messages [ 6112.347076] Lustre: server umount lustre-MDT0000 complete [ 6113.604585] LustreError: 339348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784098945 with bad export cookie 5212123293102424175 [ 6113.607368] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6113.607970] LustreError: 339348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6113.742201] Lustre: server umount lustre-MDT0001 complete [ 6124.999935] Lustre: server umount lustre-OST0000 complete [ 6135.454151] Lustre: server umount lustre-OST0001 complete [ 6136.640927] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 6137.159645] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 6137.685165] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 03:02:48 (1784098968) [ 6145.221424] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 6148.504060] loop0: detected capacity change from 0 to 1099511627776 [ 6148.655086] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 6148.659491] *** VALIDATE xfs *** [ 6148.664916] XFS (loop0): Mounting V5 Filesystem [ 6148.675434] XFS (loop0): Ending clean mount [ 6149.104728] loop1: detected capacity change from 0 to 18691697672192 [ 6170.546819] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 6173.406170] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 6184.770762] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6184.875535] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6184.886377] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6184.925850] Lustre: lustre-MDT0000: new disk, initializing [ 6184.948984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6184.951300] Lustre: Skipped 1 previous similar message [ 6184.957728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6186.391056] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6190.611816] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6190.660115] Lustre: 347489:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6190.688197] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6190.689978] Lustre: Skipped 1 previous similar message [ 6190.729502] Lustre: lustre-MDT0001: new disk, initializing [ 6190.784270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6190.788450] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6192.238733] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6196.028758] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6197.868772] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6200.191951] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6200.310332] Lustre: lustre-OST0000: new disk, initializing [ 6200.312325] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6200.314986] Lustre: 349456:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6201.592743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6201.595423] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6201.633087] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6202.392335] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6206.251603] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6207.756975] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 6207.840110] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 6209.269319] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 6209.354289] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 6212.066103] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6212.066958] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6212.068832] LustreError: Skipped 1 previous similar message [ 6212.069510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6212.074630] Lustre: Skipped 4 previous similar messages [ 6212.079173] Lustre: Skipped 5 previous similar messages [ 6217.186433] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6217.187969] Lustre: Skipped 1 previous similar message [ 6224.351097] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6224.416321] Lustre: server umount lustre-OST0000 complete [ 6226.911931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6226.914284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6226.918122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6226.919843] Lustre: Skipped 2 previous similar messages [ 6240.223115] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6240.289262] Lustre: server umount lustre-MDT0000 complete [ 6241.457333] LustreError: 348284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784099073 with bad export cookie 5212123293102425708 [ 6241.460509] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6241.461647] LustreError: 348284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6241.578525] Lustre: server umount lustre-MDT0001 complete [ 6244.036665] XFS (loop0): Unmounting Filesystem [ 6245.609733] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 03:04:36 (1784099076) [ 6247.913865] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6248.078472] LustreError: 352013:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6248.104354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6248.106081] Lustre: Skipped 2 previous similar messages [ 6249.446673] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6252.190199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6253.690737] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6256.148686] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6258.086299] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6260.203727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6262.117633] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6264.248424] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6278.114355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6278.115843] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6278.118539] Lustre: Skipped 2 previous similar messages [ 6278.118757] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6278.120140] Lustre: Skipped 7 previous similar messages [ 6283.915344] Lustre: server umount lustre-OST0000 complete [ 6288.352530] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6291.154324] Lustre: server umount lustre-MDT0000 complete [ 6292.251151] LustreError: 351995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784099123 with bad export cookie 5212123293102426590 [ 6292.253951] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6292.254925] LustreError: 351995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6292.359913] Lustre: server umount lustre-MDT0001 complete [ 6295.654307] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 6296.718415] Key type lgssc unregistered [ 6296.861526] LNet: 355658:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6296.865793] LNetError: 355658:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6296.879415] LNet: Removed LNI 192.168.202.102@tcp [ 6297.235116] Key type .llcrypt unregistered [ 6297.236212] Key type ._llcrypt unregistered [ 6308.007290] Key type ._llcrypt registered [ 6308.008418] Key type .llcrypt registered [ 6308.043459] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 6308.422278] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6308.451435] alg: No test for adler32 (adler32-zlib) [ 6309.307659] Lustre: Lustre: Build Version: 2.17.54_162_gc3b23b4 [ 6309.402658] LNet: Added LNI 192.168.202.102@tcp [8/256/0/180] [ 6310.991179] Key type lgssc registered [ 6311.362979] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6313.939517] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6313.945282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6315.093175] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6316.398958] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6319.334729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6319.467634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6320.862031] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6323.274437] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6325.149369] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6327.624899] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6327.771354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6329.645330] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6332.060109] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6345.188325] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6345.189552] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6345.193410] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6345.196537] Lustre: Skipped 1 previous similar message [ 6345.200891] Lustre: Skipped 1 previous similar message [ 6347.670538] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.2@tcp (stopping) [ 6350.308412] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6350.310345] Lustre: Skipped 1 previous similar message [ 6351.195678] Lustre: server umount lustre-MDT0001 complete [ 6352.784097] LustreError: 358632:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.2@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6352.790855] LustreError: 358632:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6354.976220] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6355.009198] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6355.010908] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6355.024115] Lustre: 357189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6355.151172] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6356.634288] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6360.550670] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6363.109852] LustreError: 360981:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6363.112601] LustreError: 360981:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6363.115350] LustreError: 357193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6363.131565] LustreError: 360984:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.102@tcp: cfg command failed: rc = -17 [ 6363.134108] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6363.134108] [ 6659.990464] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.2@tcp (stopping) [ 6662.626715] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6662.626978] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6665.103693] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.2@tcp (stopping) [ 6665.105634] Lustre: Skipped 2 previous similar messages [ 6673.375100] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6673.429116] Lustre: server umount lustre-MDT0001 complete [ 6675.343657] LustreError: 357197:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.2@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6676.915592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6676.958799] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6676.961306] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6676.980251] Lustre: 357191:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6677.108814] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6678.591937] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6682.597341] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6686.693157] LustreError: 373709:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6686.695282] LustreError: 373709:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6686.697774] LustreError: 357193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6686.699791] LustreError: 357193:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6686.716605] LustreError: 373712:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6686.718475] LustreError: 373712:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6686.720019] LustreError: 373712:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.102@tcp: cfg command failed: rc = -17 [ 6686.722248] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.102@tcp [ 6686.722248] [ 6982.549473] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.2@tcp (stopping) [ 6982.552399] Lustre: Skipped 5 previous similar messages [ 6984.675642] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6984.676081] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6984.679195] Lustre: Skipped 2 previous similar messages [ 6987.106353] Lustre: server umount lustre-MDT0001 complete [ 6987.663868] LustreError: 359408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.2@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6989.794312] LustreError: 357201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6989.799770] LustreError: 357201:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6990.616991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6990.653770] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6990.655467] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6990.668200] Lustre: 357189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6990.773216] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6992.079237] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 6995.940551] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6998.502729] LustreError: 386320:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6998.507704] LustreError: 386320:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6998.513228] LustreError: 357193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6998.515401] LustreError: 357193:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6998.536643] LustreError: 386328:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6998.542690] LustreError: 386328:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.102@tcp: cfg command failed: rc = -114 [ 6998.545352] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6998.545352] [ 7308.258988] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7308.261666] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7308.262291] Lustre: Skipped 1 previous similar message [ 7308.262535] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7308.263968] Lustre: Skipped 3 previous similar messages [ 7312.101444] Lustre: server umount lustre-MDT0000 complete [ 7313.309335] LustreError: 360405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784100144 with bad export cookie 18334705187862671282 [ 7313.313354] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7313.314644] LustreError: 360405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7313.375893] LustreError: 361287:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7313.376203] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7313.386583] Lustre: Skipped 1 previous similar message [ 7319.469329] Lustre: server umount lustre-MDT0001 complete [ 7320.757224] Lustre: server umount lustre-OST0000 complete [ 7323.688250] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 7326.841637] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 7338.167441] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 03:22:49 (1784100169) [ 7340.236716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7340.318812] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7340.328036] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7340.362775] Lustre: lustre-MDT0000: new disk, initializing [ 7340.383674] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7340.388596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7341.686695] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 7345.737382] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7345.779556] Lustre: 400942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7345.806334] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7345.808597] Lustre: Skipped 1 previous similar message [ 7345.843067] Lustre: lustre-MDT0001: new disk, initializing [ 7345.877196] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7345.891628] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7345.895174] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7347.156191] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 7350.719212] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7352.334282] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7354.491307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7354.591657] Lustre: lustre-OST0000: new disk, initializing [ 7354.594299] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7354.596512] Lustre: 402909:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7354.624830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7356.277856] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_default_debug -1 all [ 7357.044916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7357.047562] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7357.074960] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7359.708616] Lustre: DEBUG MARKER: oleg202-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7362.528940] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7362.530261] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7362.534239] Lustre: Skipped 2 previous similar messages [ 7362.537051] Lustre: Skipped 3 previous similar messages [ 7368.345097] Lustre: server umount lustre-MDT0000 complete [ 7369.472240] LustreError: 401737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784100201 with bad export cookie 18334705187862673221 [ 7369.474436] LustreError: MGC192.168.202.102@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7369.475722] LustreError: 401737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7369.479099] LustreError: 401750:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7369.588764] Lustre: server umount lustre-MDT0001 complete [ 7381.012559] Lustre: server umount lustre-OST0000 complete [ 7389.824811] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing set_hostid [ 7393.168332] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing load_modules_local [ 7402.586967] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 7232 sec ========== 03:23:53 (1784100233) [ 7403.061251] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 03:23:54 (1784100234) === [ 7404.063351] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 03:23:55 (1784100235) === [ 7416.054568] Lustre: DEBUG MARKER: oleg202-server.virtnet: executing unload_modules_local [ 7416.976727] Key type lgssc unregistered [ 7417.091319] LNet: 409266:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7417.093314] LNetError: 409266:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7417.100318] LNet: Removed LNI 192.168.202.102@tcp [ 7417.380205] Key type .llcrypt unregistered [ 7417.381225] Key type ._llcrypt unregistered