[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 426424559 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002319] x2apic enabled [ 0.003004] Switched APIC routing to physical x2apic. [ 0.004007] kvm-guest: setup PV IPIs [ 0.006707] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007009] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008004] pid_max: default: 32768 minimum: 301 [ 0.009108] LSM: Security Framework initializing [ 0.010058] Yama: becoming mindful. [ 0.011024] SELinux: Initializing. [ 0.012051] *** VALIDATE selinux *** [ 0.020649] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024703] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.025098] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.026073] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028041] *** VALIDATE tmpfs *** [ 0.029389] *** VALIDATE proc *** [ 0.030199] *** VALIDATE cgroup *** [ 0.031005] *** VALIDATE cgroup2 *** [ 0.032193] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.033135] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.034003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.036015] Spectre V2 : User space: Vulnerable [ 0.037004] Speculative Store Bypass: Vulnerable [ 0.039811] debug: unmapping init [mem 0xffffffff98a59000-0xffffffff98a60fff] [ 0.041182] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.042532] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.043011] ... version: 2 [ 0.044006] ... bit width: 48 [ 0.045006] ... generic registers: 4 [ 0.046006] ... value mask: 0000ffffffffffff [ 0.047021] ... max period: 00007fffffffffff [ 0.048006] ... fixed-purpose events: 3 [ 0.049006] ... event mask: 000000070000000f [ 0.050272] rcu: Hierarchical SRCU implementation. [ 0.052389] smp: Bringing up secondary CPUs ... [ 0.053464] x86: Booting SMP configuration: [ 0.054011] .... node #0, CPUs: #1 #2 #3 [ 0.056509] smp: Brought up 1 node, 4 CPUs [ 0.058008] smpboot: Max logical packages: 1 [ 0.059006] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.140904] node 0 deferred pages initialised in 80ms [ 0.143125] devtmpfs: initialized [ 0.144196] x86/mm: Memory block size: 128MB [ 0.146549] gcov: version magic: 0x41383552 [ 0.147550] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.148051] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.149152] pinctrl core: initialized pinctrl subsystem [ 0.150106] [ 0.150583] ************************************************************* [ 0.151007] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.152006] ** ** [ 0.153006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.154006] ** ** [ 0.155006] ** This means that this kernel is built to expose internal ** [ 0.156009] ** IOMMU data structures, which may compromise security on ** [ 0.157006] ** your system. ** [ 0.158007] ** ** [ 0.159007] ** If you see this message and you are not debugging the ** [ 0.160007] ** kernel, report this immediately to your vendor! ** [ 0.161007] ** ** [ 0.162006] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.163007] ************************************************************* [ 0.164650] NET: Registered protocol family 16 [ 0.165468] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.166034] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.167031] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.168320] cpuidle: using governor menu [ 0.169000] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.171346] PCI: Using configuration type 1 for base access [ 0.173107] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.180111] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.181022] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.182098] cryptd: max_cpu_qlen set to 1000 [ 0.183688] ACPI: Added _OSI(Module Device) [ 0.184000] ACPI: Added _OSI(Processor Device) [ 0.186009] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.187007] ACPI: Added _OSI(Processor Aggregator Device) [ 0.192443] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.198278] ACPI: Interpreter enabled [ 0.199037] ACPI: PM: (supports S0 S3 S4 S5) [ 0.200009] ACPI: Using IOAPIC for interrupt routing [ 0.201051] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.204364] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.214418] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.216026] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.218010] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.221037] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.225058] acpiphp: Slot [2] registered [ 0.226006] acpiphp: Slot [5] registered [ 0.226897] acpiphp: Slot [6] registered [ 0.228066] acpiphp: Slot [7] registered [ 0.228909] acpiphp: Slot [8] registered [ 0.230074] acpiphp: Slot [9] registered [ 0.230941] acpiphp: Slot [10] registered [ 0.232076] acpiphp: Slot [3] registered [ 0.232885] acpiphp: Slot [4] registered [ 0.233045] acpiphp: Slot [11] registered [ 0.234085] acpiphp: Slot [12] registered [ 0.235062] acpiphp: Slot [13] registered [ 0.237074] acpiphp: Slot [14] registered [ 0.238073] acpiphp: Slot [15] registered [ 0.239053] acpiphp: Slot [16] registered [ 0.240054] acpiphp: Slot [17] registered [ 0.241055] acpiphp: Slot [18] registered [ 0.242055] acpiphp: Slot [19] registered [ 0.243053] acpiphp: Slot [20] registered [ 0.244062] acpiphp: Slot [21] registered [ 0.245054] acpiphp: Slot [22] registered [ 0.246073] acpiphp: Slot [23] registered [ 0.247053] acpiphp: Slot [24] registered [ 0.248053] acpiphp: Slot [25] registered [ 0.249061] acpiphp: Slot [26] registered [ 0.251055] acpiphp: Slot [27] registered [ 0.252062] acpiphp: Slot [28] registered [ 0.253052] acpiphp: Slot [29] registered [ 0.254053] acpiphp: Slot [30] registered [ 0.255053] acpiphp: Slot [31] registered [ 0.256048] PCI host bridge to bus 0000:00 [ 0.257010] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.259012] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.260008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.262011] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.264010] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.266010] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.267122] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.269874] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.272039] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.281011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.285675] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.287009] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.289008] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.291008] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.293488] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.296047] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.298024] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.300888] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.307011] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.315965] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.318006] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.321215] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.326009] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.332008] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.342013] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.350452] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.354008] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.358007] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.368011] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.375009] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.379009] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.385012] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.400016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.408654] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.413008] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.416007] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.426013] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.432286] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.438009] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.441009] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.452013] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.460885] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.465008] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.470009] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.482016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.491949] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.493248] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.495190] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.496185] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.498104] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.502040] iommu: Default domain type: Passthrough [ 0.504526] SCSI subsystem initialized [ 0.505072] ACPI: bus type USB registered [ 0.505943] usbcore: registered new interface driver usbfs [ 0.507039] usbcore: registered new interface driver hub [ 0.508036] usbcore: registered new device driver usb [ 0.509094] pps_core: LinuxPPS API ver. 1 registered [ 0.510006] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.512019] PTP clock support registered [ 0.514035] EDAC MC: Ver: 3.0.0 [ 0.515209] PCI: Using ACPI for IRQ routing [ 0.516778] NetLabel: Initializing [ 0.517011] NetLabel: domain hash size = 128 [ 0.518010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.519097] NetLabel: unlabeled traffic allowed by default [ 0.521104] vgaarb: loaded [ 0.522255] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.523006] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.531242] clocksource: Switched to clocksource kvm-clock [ 0.618913] VFS: Disk quotas dquot_6.6.0 [ 0.619868] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.621371] *** VALIDATE ramfs *** [ 0.622149] *** VALIDATE hugetlbfs *** [ 0.623119] pnp: PnP ACPI init [ 0.624706] pnp: PnP ACPI: found 6 devices [ 0.637893] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.641700] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.643623] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.644960] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.646304] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.647668] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.649402] NET: Registered protocol family 2 [ 0.651123] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.654348] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.656525] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.660587] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.662678] TCP: Hash tables configured (established 65536 bind 65536) [ 0.664422] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.666253] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.667928] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.669577] NET: Registered protocol family 1 [ 0.671226] RPC: Registered named UNIX socket transport module. [ 0.672616] RPC: Registered udp transport module. [ 0.673690] RPC: Registered tcp transport module. [ 0.674770] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.676132] NET: Registered protocol family 44 [ 0.677182] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.678513] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.679960] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.681644] PCI: CLS 0 bytes, default 64 [ 0.684134] Unpacking initramfs... [ 1.931579] debug: unmapping init [mem 0xffff99babcc54000-0xffff99babffbffff] [ 1.934246] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 1.935645] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 1.937588] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.483109] Initialise system trusted keyrings [ 2.485099] Key type blacklist registered [ 2.487204] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.497686] zbud: loaded [ 2.500921] *** VALIDATE nfs *** [ 2.502279] *** VALIDATE nfs4 *** [ 2.503979] pstore: using deflate compression [ 2.506770] Platform Keyring initialized [ 2.586614] NET: Registered protocol family 38 [ 2.588336] Key type asymmetric registered [ 2.589805] Asymmetric key parser 'x509' registered [ 2.591555] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.594323] io scheduler mq-deadline registered [ 2.595969] io scheduler kyber registered [ 2.597570] io scheduler bfq registered [ 2.599274] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.602069] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.604758] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.607497] ACPI: Power Button [PWRF] [ 2.612738] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.619494] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.632471] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.639035] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.648389] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.673830] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.708903] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.718794] Non-volatile memory driver v1.3 [ 2.720537] Linux agpgart interface v0.103 [ 2.750207] virtio_blk virtio1: [vda] 139280 512-byte logical blocks (71.3 MB/68.0 MiB) [ 2.754018] vda: detected capacity change from 0 to 71311360 [ 2.764628] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.767024] vdb: detected capacity change from 0 to 1073741824 [ 2.778133] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.780379] vdc: detected capacity change from 0 to 2621440000 [ 2.791280] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.793486] vdd: detected capacity change from 0 to 2621440000 [ 2.804261] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.806474] vde: detected capacity change from 0 to 4294967296 [ 2.817776] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.820009] vdf: detected capacity change from 0 to 4294967296 [ 2.825069] libphy: Fixed MDIO Bus: probed [ 2.829076] usbcore: registered new interface driver usbserial_generic [ 2.831936] usbserial: USB Serial support registered for generic [ 2.834699] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.839460] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.842602] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.845396] mousedev: PS/2 mouse device common for all mice [ 2.847945] rtc_cmos 00:05: RTC can wake from S4 [ 2.849539] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.850433] rtc_cmos 00:05: registered as rtc0 [ 2.853759] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.858397] intel_pstate: CPU model not supported [ 2.861139] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.863603] hid: raw HID events driver (C) Jiri Kosina [ 2.866792] usbcore: registered new interface driver usbhid [ 2.868983] usbhid: USB HID core driver [ 2.870616] drop_monitor: Initializing network drop monitor service [ 2.870629] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.873119] Initializing XFRM netlink socket [ 2.878747] NET: Registered protocol family 10 [ 2.880792] Segment Routing with IPv6 [ 2.881793] NET: Registered protocol family 17 [ 2.883081] mpls_gso: MPLS GSO support [ 2.889250] RAS: Correctable Errors collector initialized. [ 2.890595] AVX version of gcm_enc/dec engaged. [ 2.891820] AES CTR mode by8 optimization enabled [ 2.938587] sched_clock: Marking stable (2938565322, 0)->(3829853626, -891288304) [ 2.941764] registered taskstats version 1 [ 2.943498] Loading compiled-in X.509 certificates [ 2.945363] zswap: loaded using pool lzo/zbud [ 2.963362] Key type big_key registered [ 2.972109] Key type encrypted registered [ 2.973380] ima: No TPM chip found, activating TPM-bypass! [ 2.975058] ima: Allocated hash algorithm: sha1 [ 2.976245] ima: No architecture policies found [ 2.977506] evm: Initialising EVM extended attributes: [ 2.978932] evm: security.selinux [ 2.979925] evm: security.ima [ 2.980644] evm: security.capability [ 2.981586] evm: HMAC attrs: 0x1 [ 2.983218] rtc_cmos 00:05: setting system clock to 2026-06-16 11:21:35 UTC (1781608895) [ 2.987692] debug: unmapping init [mem 0xffffffff99a03000-0xffffffff99bfffff] [ 2.989789] debug: unmapping init [mem 0xffffffff98782000-0xffffffff98a58fff] [ 2.999074] Write protecting the kernel read-only data: 28672k [ 3.001461] debug: unmapping init [mem 0xffffffff96e03000-0xffffffff96ffffff] [ 3.004047] debug: unmapping init [mem 0xffffffff97714000-0xffffffff977fffff] [ 3.032336] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.040812] systemd[1]: Detected virtualization kvm. [ 3.042376] systemd[1]: Detected architecture x86-64. [ 3.043643] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.069122] systemd[1]: No hostname configured. [ 3.071238] systemd[1]: Set hostname to . [ 3.073461] random: systemd: uninitialized urandom read (16 bytes read) [ 3.075979] systemd[1]: Initializing machine ID from random generator. [ 3.128777] random: ln: uninitialized urandom read (6 bytes read) [ 3.202753] random: systemd: uninitialized urandom read (16 bytes read) [ 3.205167] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.207832] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.210644] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.703095] device-mapper: uevent: version 1.0.3 [ 3.705274] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 4.338172] virtio_net virtio0 ens2: renamed from eth0 [ 4.374273] random: fast init done [ 4.381905] scsi host0: ata_piix [ 4.420204] scsi host1: ata_piix [ 4.422122] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.425512] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.133288] dracut-initqueue[589]: RTNETLINK answers: File exists [ 9.485207] random: crng init done [ 9.486761] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 9.709101] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.663120] printk: systemd: 25 output lines suppressed due to ratelimiting [ 10.891556] SELinux: Disabled at runtime. [ 10.951330] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 10.959854] systemd[1]: Detected virtualization kvm. [ 10.961359] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.363499] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.366481] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.372453] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.376506] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.379708] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.391086] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.408889] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. Mounting POSIX Message Queue File System... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 11.461038] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Starting Apply Kernel Variables... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd Root File System. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 11.793166] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 12.067683] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.074658] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.164255] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.180991] EDAC sbridge: Ver: 1.1.2 [ 13.511863] Key type dns_resolver registered [ 13.772360] NFS: Registering the id_resolver key type [ 13.773466] Key type id_resolver registered [ 13.774340] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg213-server login: [ 34.020111] libcfs: loading out-of-tree module taints kernel. [ 34.036570] Key type ._llcrypt registered [ 34.037569] Key type .llcrypt registered [ 34.074882] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 40.934245] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 41.614067] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 41.619162] alg: No test for adler32 (adler32-zlib) [ 42.570067] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 42.840511] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 44.439149] Key type lgssc registered [ 44.971546] Lustre: Echo OBD driver; http://www.lustre.org/ [ 50.895563] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 63.841249] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 68.092104] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 68.099546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 69.209737] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 69.222264] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 69.261489] Lustre: lustre-MDT0000: new disk, initializing [ 69.285963] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 69.292503] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 70.585309] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 75.920875] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 75.956099] Lustre: 6484:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 75.968490] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 75.971891] Lustre: Skipped 1 previous similar message [ 76.010751] Lustre: lustre-MDT0001: new disk, initializing [ 76.028431] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 76.035305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 76.038243] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 77.288174] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 79.488824] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 82.665809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 82.752623] Lustre: lustre-OST0000: new disk, initializing [ 82.754189] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 82.756344] Lustre: 8423:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 82.772393] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 84.612192] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 84.713723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 84.716333] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 84.734579] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 90.138268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 90.187950] Lustre: lustre-OST0001: new disk, initializing [ 90.190772] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 90.194559] Lustre: 9496:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 90.213468] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 92.277836] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 92.463987] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 92.468717] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 92.492297] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 98.229775] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 101.825518] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 103.685954] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing check_logdir /tmp/testlogs/ [ 105.229359] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing yml_node [ 106.571504] Lustre: DEBUG MARKER: Client: 2.17.54.40 [ 107.384308] Lustre: DEBUG MARKER: MDS: 2.17.54.40 [ 108.176220] Lustre: DEBUG MARKER: OSS: 2.17.54.40 [ 108.685112] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Jun 16 07:23:20 EDT 2026 [ 113.975789] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 114.441065] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 127.455501] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 127.457884] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 127.461813] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 128.479515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 128.480178] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 128.481595] Lustre: Skipped 1 previous similar message [ 128.484927] Lustre: Skipped 1 previous similar message [ 132.252378] Lustre: server umount lustre-MDT0000 complete [ 133.470810] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609025 with bad export cookie 13653595883791016283 [ 133.471905] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 133.474486] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 133.599639] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 133.600080] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 133.603281] Lustre: Skipped 2 previous similar messages [ 133.607068] Lustre: Skipped 1 previous similar message [ 138.719724] LustreError: 7802:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 138.719896] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 138.725087] LustreError: 7802:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 138.729520] Lustre: Skipped 1 previous similar message [ 139.619941] Lustre: server umount lustre-MDT0001 complete [ 155.103107] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 155.136137] Lustre: server umount lustre-OST0000 complete [ 162.448335] Lustre: server umount lustre-OST0001 complete [ 164.579869] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 167.665737] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 179.603481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 179.687856] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 179.698608] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 179.734207] Lustre: lustre-MDT0000: new disk, initializing [ 179.759267] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 179.767489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 180.973474] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 184.771414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 184.808585] Lustre: 15040:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 184.902273] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 186.029681] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 189.480713] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 191.173210] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 193.215195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 193.306195] Lustre: lustre-OST0000: new disk, initializing [ 193.307366] Lustre: Skipped 1 previous similar message [ 193.308668] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 193.310323] Lustre: Skipped 2 previous similar messages [ 193.312364] Lustre: 17004:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 193.339171] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.342409] Lustre: Skipped 1 previous similar message [ 195.077810] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 195.255746] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 195.257794] Lustre: Skipped 1 previous similar message [ 195.259913] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 195.289685] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 198.467129] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 199.658124] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 199.718152] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 200.904787] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 200.965597] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 205.793442] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 205.793973] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 205.794548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 205.796768] Lustre: Skipped 1 previous similar message [ 205.798658] LustreError: Skipped 1 previous similar message [ 207.627943] Lustre: server umount lustre-OST0000 complete [ 210.399696] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 210.401872] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 216.031917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 216.033670] Lustre: Skipped 4 previous similar messages [ 223.199131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 223.252259] Lustre: server umount lustre-MDT0000 complete [ 224.351961] LustreError: 15032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609116 with bad export cookie 13653595883791017802 [ 224.354117] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 224.355015] LustreError: 15032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 224.455743] Lustre: server umount lustre-MDT0001 complete [ 225.756273] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 07:25:17 (1781609117) [ 227.685313] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 227.812166] LustreError: 19268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 227.831979] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 228.952371] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 231.511420] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 232.782919] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 234.828629] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 236.375211] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 238.496227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 240.424218] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 242.731969] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 255.207561] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 262.623853] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 262.626483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 262.630192] Lustre: Skipped 1 previous similar message [ 262.631954] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 262.634119] Lustre: Skipped 2 previous similar messages [ 264.408741] Lustre: server umount lustre-MDT0000 complete [ 265.621455] LustreError: 21819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609158 with bad export cookie 13653595883791018684 [ 265.624638] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 265.627848] LustreError: 21819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 265.740820] Lustre: server umount lustre-MDT0001 complete [ 277.206369] Lustre: server umount lustre-OST0000 complete [ 284.246102] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 287.798367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 287.937582] LustreError: 23665:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 287.954687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 287.957052] Lustre: Skipped 2 previous similar messages [ 289.295700] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 292.324772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 293.779763] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 294.749402] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 296.973482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 298.968630] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 302.056186] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 302.117763] Lustre: lustre-OST0001: new disk, initializing [ 302.119393] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 302.121601] Lustre: 25978:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 303.616322] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 303.621741] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 303.653936] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 304.017201] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 308.586822] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 310.933838] Lustre: Setting parameter general.obdfilter.*.client_cache_count=256 in log params [ 310.935858] Lustre: Skipped 1 previous similar message [ 313.826370] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 313.828130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 313.830645] Lustre: Skipped 4 previous similar messages [ 313.832545] Lustre: Skipped 4 previous similar messages [ 319.838496] Lustre: server umount lustre-MDT0000 complete [ 321.086255] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609213 with bad export cookie 13653595883791019531 [ 321.089955] LustreError: 23645:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 321.090247] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 321.208747] Lustre: server umount lustre-MDT0001 complete [ 333.315064] Lustre: server umount lustre-OST0000 complete [ 343.731495] Lustre: server umount lustre-OST0001 complete [ 349.136698] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 352.422136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 352.561146] LustreError: 28993:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 352.578417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 352.580234] Lustre: Skipped 3 previous similar messages [ 353.917209] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 356.514210] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 357.914717] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 358.808455] Lustre: 30137:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 360.814591] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 362.644909] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 362.980959] LustreError: 30490:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.201993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 366.947842] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 369.869607] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 370.934541] Lustre: 32012:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 372.707393] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.711439] Lustre: Skipped 3 previous similar messages [ 377.311873] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 377.828191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 377.829819] Lustre: Skipped 12 previous similar messages [ 378.583967] Lustre: server umount lustre-MDT0000 complete [ 379.721116] LustreError: 32015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609272 with bad export cookie 13653595883791020763 [ 379.724277] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 379.725687] LustreError: 32015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 379.841950] Lustre: server umount lustre-MDT0001 complete [ 391.173986] Lustre: server umount lustre-OST0000 complete [ 401.630056] Lustre: server umount lustre-OST0001 complete [ 404.454780] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 07:28:16 (1781609296) [ 413.287726] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 416.469871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 416.613072] LustreError: 34811:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 417.895145] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 420.569442] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 422.021437] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 422.999376] Lustre: 35958:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 425.128515] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 427.110371] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 429.883674] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 431.805144] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 434.857096] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 436.068316] Lustre: 37832:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 441.316162] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 441.316644] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 441.322696] Lustre: Skipped 4 previous similar messages [ 443.417830] Lustre: server umount lustre-MDT0000 complete [ 444.599472] LustreError: 34792:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609337 with bad export cookie 13653595883791021862 [ 444.605670] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 444.732941] Lustre: server umount lustre-MDT0001 complete [ 455.619976] Lustre: server umount lustre-OST0000 complete [ 467.030794] Lustre: server umount lustre-OST0001 complete [ 469.777513] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 07:29:21 (1781609361) [ 474.561367] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 477.710808] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 477.841583] LustreError: 39969:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 477.846882] LustreError: 39969:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 479.087168] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 481.774961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 481.909722] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 481.913384] Lustre: Skipped 8 previous similar messages [ 483.186739] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 484.124344] Lustre: 41119:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 486.290049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 488.223821] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 490.971706] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 492.919250] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 495.985876] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 497.380986] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 500.437756] Lustre: server umount lustre-MDT0000 complete [ 501.660765] LustreError: 39950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609394 with bad export cookie 13653595883791022947 [ 501.664383] LustreError: 39950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 501.666866] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 506.272408] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 506.274230] Lustre: Skipped 9 previous similar messages [ 507.815919] Lustre: server umount lustre-MDT0001 complete [ 510.394153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 512.479086] LustreError: 43847:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 512.479553] LustreError: 3619:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff99bb2dec9180 x1868152370907520/t0(0) o250->MGC192.168.202.113@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 512.584528] LustreError: 41482:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 512.589647] LustreError: 41482:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 513.819162] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 516.429075] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.818164] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 520.126070] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 529.790587] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 9 sec [ 530.920876] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 531.580285] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 532.130953] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 539.549056] Lustre: server umount lustre-MDT0000 complete [ 540.742378] LustreError: 39950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609433 with bad export cookie 13653595883791024025 [ 540.747319] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 540.862518] Lustre: server umount lustre-MDT0001 complete [ 552.389831] Lustre: server umount lustre-OST0000 complete [ 563.873667] Lustre: server umount lustre-OST0001 complete [ 566.697838] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 07:30:58 (1781609458) [ 571.858366] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 575.044301] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 576.468095] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 579.209553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 580.608408] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 581.495921] Lustre: 48739:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 581.498415] Lustre: 48739:0:(mgs_llog.c:1447:mgs_modify_param()) Skipped 1 previous similar message [ 583.539199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 584.676567] LustreError: 49092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 584.681804] LustreError: 49092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 585.582701] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 588.347113] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 590.319806] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 593.352034] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 594.658739] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 600.036548] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 600.037546] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 600.041318] Lustre: Skipped 10 previous similar messages [ 604.058163] Lustre: server umount lustre-MDT0000 complete [ 605.233210] LustreError: 47570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609497 with bad export cookie 13653595883791024851 [ 605.236360] LustreError: 47570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 605.237074] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 605.347951] Lustre: server umount lustre-MDT0001 complete [ 617.469832] Lustre: server umount lustre-OST0000 complete [ 628.951141] Lustre: server umount lustre-OST0001 complete [ 631.996761] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:32:03 (1781609523) [ 632.822109] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 633.414619] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:32:05 (1781609525) [ 636.660325] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 639.454079] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 646.541945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 646.645261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 646.683141] Lustre: lustre-MDT0000: new disk, initializing [ 646.712929] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 647.972735] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 651.562851] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 653.811539] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 653.906542] Lustre: 54803:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 655.040682] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 655.051019] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 655.752992] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 659.276851] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 659.975206] Lustre: DEBUG MARKER: create test files [ 675.808915] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 676.937078] Lustre: server umount lustre-OST0000 complete [ 678.210989] Lustre: server umount lustre-MDT0000 complete [ 681.711761] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 682.646606] Key type lgssc unregistered [ 682.766804] LNet: 56430:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 682.769320] LNetError: 56430:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 682.777326] LNet: Removed LNI 192.168.202.113@tcp [ 683.088112] Key type .llcrypt unregistered [ 683.089350] Key type ._llcrypt unregistered [ 693.492025] Key type ._llcrypt registered [ 693.493113] Key type .llcrypt registered [ 693.529747] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 693.991822] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 693.997250] alg: No test for adler32 (adler32-zlib) [ 694.847550] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 694.925024] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 696.503174] Key type lgssc registered [ 696.840673] Lustre: Echo OBD driver; http://www.lustre.org/ [ 699.319762] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 699.326444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 700.473392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 701.741534] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 704.106125] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 706.357745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 706.469263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 708.325111] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 709.541552] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 709.564282] Lustre: 59134:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 709.567131] Lustre: 59134:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 709.569185] Lustre: 59134:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 709.571361] Lustre: 59134:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 709.573557] Lustre: 59134:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 709.575968] Lustre: 59134:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 710.663619] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 713.630995] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 715.150325] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 719.841830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 719.841927] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 719.845835] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 723.021740] Lustre: server umount lustre-OST0000 complete [ 724.353737] Lustre: server umount lustre-MDT0000 complete [ 727.760829] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 728.703423] Key type lgssc unregistered [ 728.825660] LNet: 60744:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 728.827801] LNetError: 60744:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 728.838292] LNet: Removed LNI 192.168.202.113@tcp [ 729.109737] Key type .llcrypt unregistered [ 729.110799] Key type ._llcrypt unregistered [ 739.813685] Key type ._llcrypt registered [ 739.814650] Key type .llcrypt registered [ 739.855674] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 740.307553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 740.318232] alg: No test for adler32 (adler32-zlib) [ 741.166765] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 741.245238] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 742.823170] Key type lgssc registered [ 743.175300] Lustre: Echo OBD driver; http://www.lustre.org/ [ 745.598441] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 745.604843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 746.750850] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 748.066873] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 750.311073] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 752.573423] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 752.703880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 754.530434] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 756.836761] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 756.858045] Lustre: 63550:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 756.861710] Lustre: 63550:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 756.864039] Lustre: 63550:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 756.866302] Lustre: 63550:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 756.869270] Lustre: 63550:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 756.871539] Lustre: 63550:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 756.924059] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 759.910338] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 762.336649] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 762.336810] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 762.340580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 764.239834] Lustre: server umount lustre-OST0000 complete [ 767.603472] Lustre: server umount lustre-MDT0000 complete [ 771.230698] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 772.227293] Key type lgssc unregistered [ 772.354380] LNet: 65108:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 772.357473] LNetError: 65108:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 772.366274] LNet: Removed LNI 192.168.202.113@tcp [ 772.650397] Key type .llcrypt unregistered [ 772.651561] Key type ._llcrypt unregistered [ 778.423805] Key type ._llcrypt registered [ 778.424922] Key type .llcrypt registered [ 778.460369] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 784.827395] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 785.164598] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 785.195840] alg: No test for adler32 (adler32-zlib) [ 786.053123] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 786.137976] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 787.719112] Key type lgssc registered [ 788.130985] Lustre: Echo OBD driver; http://www.lustre.org/ [ 799.216897] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 799.222057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 800.312990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 800.327317] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 800.374358] Lustre: lustre-MDT0000: new disk, initializing [ 800.400435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 800.407089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 801.757374] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 805.932276] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 805.975658] Lustre: 68244:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 806.003418] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 806.006971] Lustre: Skipped 1 previous similar message [ 806.043751] Lustre: lustre-MDT0001: new disk, initializing [ 806.082073] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 806.094462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 806.097696] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 807.379595] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 811.021424] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 812.815914] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 815.210656] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 815.346870] Lustre: lustre-OST0000: new disk, initializing [ 815.348968] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 815.351866] Lustre: 70213:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 815.381715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 815.793221] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 815.796578] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 815.826440] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 817.352358] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 820.986467] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 822.354858] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 822.423838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 823.850299] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 823.919534] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 826.337250] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 826.338676] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 826.339874] LustreError: Skipped 1 previous similar message [ 826.344230] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 826.346508] Lustre: Skipped 1 previous similar message [ 830.669656] Lustre: server umount lustre-OST0000 complete [ 831.968083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 831.970379] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 831.973950] Lustre: Skipped 1 previous similar message [ 831.975847] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 836.576854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 836.577590] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 841.696054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 841.698580] Lustre: Skipped 2 previous similar messages [ 845.792502] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 845.795073] Lustre: Skipped 1 previous similar message [ 846.303064] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 846.382029] Lustre: server umount lustre-MDT0000 complete [ 847.628456] LustreError: 68239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609740 with bad export cookie 11518308199439950657 [ 847.632179] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 847.632748] LustreError: 68239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 847.756780] Lustre: server umount lustre-MDT0001 complete [ 850.447493] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:35:42 (1781609742) [ 854.118144] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 866.631485] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 869.679507] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 881.643864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 881.736682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 881.758508] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 881.794459] Lustre: lustre-MDT0000: new disk, initializing [ 881.817160] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 881.822891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 883.101317] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 887.153141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 887.198361] Lustre: 76222:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 887.227189] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 887.229268] Lustre: Skipped 1 previous similar message [ 887.264645] Lustre: lustre-MDT0001: new disk, initializing [ 887.303535] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 887.317651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 887.321744] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 888.679812] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 892.394309] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 894.183781] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 896.419947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 896.515758] Lustre: lustre-OST0000: new disk, initializing [ 896.517451] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 896.520187] Lustre: 78192:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 896.547711] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 897.209549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 897.212889] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 897.243805] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 898.485795] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 902.088273] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 903.477615] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 903.549866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 904.876680] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 904.946602] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 907.600183] Lustre: server umount lustre-OST0000 complete [ 912.865128] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 912.865216] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 912.865888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 912.868613] Lustre: Skipped 1 previous similar message [ 923.103097] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 923.168522] Lustre: server umount lustre-MDT0000 complete [ 924.128080] Lustre: 66473:0:(client.c:2489:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781609800/real 1781609800] req@00000000cba33bb8 x1868153149777152/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1781609816 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 924.136030] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 924.376106] LustreError: 77015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609816 with bad export cookie 11518308199439951525 [ 924.378978] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 924.379844] LustreError: 77015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 924.503265] Lustre: server umount lustre-MDT0001 complete [ 927.302322] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:36:59 (1781609819) [ 929.496348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 929.637767] LustreError: 80600:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 929.645259] LustreError: 80600:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 929.659053] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 930.996570] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 933.819651] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 935.235699] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 937.645814] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 939.438120] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 941.667752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 943.648378] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 945.986250] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 947.211629] Lustre: 83268:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 14c56d62-ddc5-4a87-8797-269f7f9043e5 at adminstrative request [ 949.223635] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 949.227197] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 959.460156] Lustre: MGS: Client 14c56d62-ddc5-4a87-8797-269f7f9043e5 (at 0@lo) reconnecting [ 959.464209] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x9fd94047e1166a09 to 0x9fd94047e1166c94 [ 959.470763] Lustre: MGC192.168.202.113@tcp: Connection restored to 0@lo (at 0@lo) [ 982.347883] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 982.499641] Lustre: lustre-OST0001: new disk, initializing [ 982.502531] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 982.508206] Lustre: 83660:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 982.633154] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 982.642041] Lustre: Skipped 2 previous similar messages [ 987.663910] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 987.699689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 987.705438] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 987.800882] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 993.920937] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 997.855910] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 997.859977] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 997.867145] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 997.869414] Lustre: Skipped 5 previous similar messages [ 1001.428301] Lustre: server umount lustre-OST0001 complete [ 1003.495682] LustreError: 83290:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1003.505539] LustreError: 83290:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1008.612442] LustreError: 82421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1008.624125] LustreError: 82421:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1009.448119] Lustre: server umount lustre-OST0000 complete [ 1017.318923] Lustre: server umount lustre-MDT0000 complete [ 1018.742257] LustreError: 81335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609911 with bad export cookie 11518308199439953044 [ 1018.745131] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1018.747083] LustreError: 81335:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1018.866575] Lustre: server umount lustre-MDT0001 complete [ 1021.700835] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 07:38:33 (1781609913) [ 1022.240805] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1022.784592] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 07:38:34 (1781609914) [ 1023.266282] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1023.811815] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 07:38:35 (1781609915) [ 1024.343029] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1024.917809] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 07:38:36 (1781609916) [ 1025.552886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1026.189079] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 07:38:38 (1781609918) [ 1026.742864] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1028.910242] loop: module loaded [ 1028.914199] loop0: detected capacity change from 0 to 4294967296 [ 1028.937810] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1029.475212] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1030.486266] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1031.834380] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1041.596619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1041.608869] LustreError: 87778:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1041.611581] LustreError: 87778:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1041.613966] LustreError: 87778:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1041.616680] LustreError: 87778:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1048.594642] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 1051.970694] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1063.873586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1063.965223] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1063.977722] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1064.014311] Lustre: lustre-MDT0000: new disk, initializing [ 1064.040893] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1064.048466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1065.350466] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1069.329289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1069.364930] Lustre: 90638:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1069.455045] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1070.826555] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1074.538758] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1076.444417] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1078.670211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1078.766934] Lustre: 92604:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1079.457209] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1080.726420] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1084.467456] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1085.918180] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1085.986797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1087.457174] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1087.532430] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1090.018766] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1090.018873] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1090.022971] Lustre: Skipped 5 previous similar messages [ 1090.025802] LustreError: Skipped 1 previous similar message [ 1090.028602] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1090.032141] Lustre: Skipped 9 previous similar messages [ 1094.283805] Lustre: server umount lustre-OST0000 complete [ 1100.257942] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1101.668899] Lustre: server umount lustre-MDT0000 complete [ 1102.951214] LustreError: 90630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781609995 with bad export cookie 11518308199439953667 [ 1102.953820] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1102.956109] LustreError: 90630:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1103.075737] Lustre: server umount lustre-MDT0001 complete [ 1106.334527] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 07:39:58 (1781609998) [ 1108.891213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1109.055080] LustreError: 95009:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1110.567995] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1113.724866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1115.292954] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1117.974225] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1119.818933] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1122.107354] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1124.105111] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1126.295556] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1127.603077] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1134.832356] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 7 sec [ 1136.180879] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1136.252891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1138.480485] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1138.541587] Lustre: lustre-OST0001: new disk, initializing [ 1138.543229] Lustre: Skipped 2 previous similar messages [ 1138.545122] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1138.547241] Lustre: Skipped 3 previous similar messages [ 1138.549410] Lustre: 98425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1140.595941] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1144.320690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1144.323627] Lustre: Skipped 2 previous similar messages [ 1144.325916] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1144.328367] Lustre: Skipped 1 previous similar message [ 1144.356118] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1144.383257] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1145.776523] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1145.845846] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1147.354407] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1147.420926] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1157.593861] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1162.955704] Lustre: Failing over lustre-MDT0000 [ 1163.239175] Lustre: server umount lustre-MDT0000 complete [ 1164.248496] LustreError: 95004:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1164.255207] LustreError: 95004:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1164.773435] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1164.776854] Lustre: Skipped 6 previous similar messages [ 1174.484525] LustreError: 95006:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1174.490625] LustreError: 95006:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 1177.924732] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1177.925986] LDISKFS-fs (dm-0): recovery complete [ 1177.928658] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1177.969447] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1178.106659] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1179.491114] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1179.603189] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1179.605398] Lustre: 95005:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1183.204504] Lustre: 95010:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1183.206350] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1183.207206] Lustre: 95010:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 1183.225331] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1183.543075] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1183.865236] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1183.867906] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1184.183139] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1184.186777] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 1184.505388] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1184.508975] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 8 previous similar messages [ 1185.137400] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1185.139920] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1185.455099] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1185.457612] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 3 previous similar messages [ 1186.711951] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1186.715730] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 1187.343846] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1187.347913] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1187.663081] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1187.666315] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 1190.793413] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1190.797876] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 1191.425364] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1191.428839] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1191.743127] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1191.745528] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 1198.929397] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1198.932487] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 1199.562616] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1199.569528] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1199.887144] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1199.890585] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 1215.184982] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1215.188132] Lustre: 101244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 51 previous similar messages [ 1215.817419] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1215.821379] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1216.135096] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1216.137653] LustreError: 101244:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 51 previous similar messages [ 1239.255122] Lustre: 101244:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1239.257586] Lustre: 101244:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1239.259812] Lustre: 101244:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ce5f9a53-bac0-4222-be3f-8d9cf849db9f@192.168.202.13@tcp [ 1239.262951] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1239.265417] Lustre: 101244:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000de15a095 x1868153143494272/t0(8589934776) o35->ce5f9a53-bac0-4222-be3f-8d9cf849db9f@192.168.202.13@tcp:402/0 lens 392/0 e 0 to 0 dl 1781610142 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1239.272810] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1239.275559] Lustre: lustre-MDT0000: Denying connection for new client ce5f9a53-bac0-4222-be3f-8d9cf849db9f (at 192.168.202.13@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1239.278833] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1239.293571] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1239.293753] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1239.295660] Lustre: Skipped 2 previous similar messages [ 1239.314302] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1239.314348] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 1245.152126] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1245.156292] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1245.161120] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1245.163246] Lustre: Skipped 2 previous similar messages [ 1251.260191] Lustre: server umount lustre-OST0000 complete [ 1254.884118] LustreError: 99119:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1254.890284] LustreError: 99119:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 1258.639625] Lustre: server umount lustre-OST0001 complete [ 1274.335161] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1274.424754] Lustre: server umount lustre-MDT0000 complete [ 1275.632389] LustreError: 94989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610168 with bad export cookie 11518308199440004851 [ 1275.636800] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1275.637482] LustreError: 94989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1275.770816] Lustre: server umount lustre-MDT0001 complete [ 1278.496593] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:42:50 (1781610170) [ 1280.999871] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1281.003861] Lustre: *** cfs_fail_loc=197, val=0*** [ 1296.415514] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1301.535348] LustreError: 103215:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.113@tcp: failed processing log, type 4: rc = -110 [ 1306.663224] Lustre: *** cfs_fail_loc=197, val=0*** [ 1306.665200] Lustre: Skipped 65 previous similar messages [ 1327.135417] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1327.137679] Lustre: Skipped 7 previous similar messages [ 1328.924999] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1331.173188] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1331.889220] Lustre: server umount lustre-OST0000 complete [ 1334.652838] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:43:46 (1781610226) [ 1348.355582] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1348.514174] LustreError: 104834:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1348.518857] LustreError: 104834:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1349.896911] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1353.567312] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1355.554890] LustreError: 104868:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1355.558409] LustreError: 104868:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1355.561800] LustreError: 104868:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1355.684472] Lustre: server umount lustre-MDT0000 complete [ 1366.159686] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 1369.607497] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1381.984644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1382.076653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1382.090926] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1382.134496] Lustre: lustre-MDT0000: new disk, initializing [ 1382.161263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1383.547086] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1387.764850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1387.807248] Lustre: 109031:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1387.916842] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1389.208210] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1393.057899] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1394.830962] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1397.605342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1397.747361] Lustre: 110998:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1399.141019] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1399.746226] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1403.510242] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1404.970382] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1405.033854] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1406.357977] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1406.427594] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1409.505993] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1409.506564] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1409.507789] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1409.507793] Lustre: Skipped 11 previous similar messages [ 1409.510264] Lustre: Skipped 6 previous similar messages [ 1409.512982] LustreError: Skipped 3 previous similar messages [ 1421.279157] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1421.333897] Lustre: server umount lustre-OST0000 complete [ 1428.829965] Lustre: server umount lustre-MDT0000 complete [ 1429.983921] LustreError: 109832:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1429.988147] LustreError: 109832:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1430.023076] LustreError: 109826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610322 with bad export cookie 11518308199440015498 [ 1430.026356] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1430.029260] LustreError: 109826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1430.138893] Lustre: server umount lustre-MDT0001 complete [ 1432.910770] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:45:24 (1781610324) [ 1435.833563] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 1436.824611] Key type lgssc unregistered [ 1436.942342] LNet: 113483:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1436.944591] LNetError: 113483:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1436.953283] LNet: Removed LNI 192.168.202.113@tcp [ 1437.236311] Key type .llcrypt unregistered [ 1437.237348] Key type ._llcrypt unregistered [ 1443.197022] Key type ._llcrypt registered [ 1443.198081] Key type .llcrypt registered [ 1443.232785] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 1449.412336] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1449.716883] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1449.742138] alg: No test for adler32 (adler32-zlib) [ 1450.606868] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 1450.696235] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 1452.271202] Key type lgssc registered [ 1452.684666] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1465.917377] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1465.924525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1467.036779] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1467.048854] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1467.088146] Lustre: lustre-MDT0000: new disk, initializing [ 1467.114062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1467.121238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1468.532339] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1472.674555] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1472.792867] Lustre: lustre-OST0000: new disk, initializing [ 1472.795312] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1472.797836] Lustre: Skipped 1 previous similar message [ 1472.800386] Lustre: 117630:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1472.825571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1472.950659] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1472.953357] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1472.960982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1474.877041] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1479.070617] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1479.136171] Lustre: lustre-OST0001: new disk, initializing [ 1479.139259] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1479.141846] Lustre: 118493:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1479.170804] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1479.816896] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1479.819750] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1479.827913] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1481.079995] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1485.280961] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1485.281767] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1485.289690] Lustre: Skipped 1 previous similar message [ 1490.400231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1490.403039] Lustre: Skipped 2 previous similar messages [ 1491.241142] Lustre: server umount lustre-MDT0000 complete [ 1493.294612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1494.758154] LustreError: 116815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610387 with bad export cookie 7191839697657391325 [ 1494.760349] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1494.761787] LustreError: 116815:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1494.806977] Lustre: server umount lustre-OST0000 complete [ 1496.066221] Lustre: server umount lustre-OST0001 complete [ 1498.770753] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 07:46:30 (1781610390) [ 1501.945133] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1509.505518] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:46:41 (1781610401) [ 1522.107114] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 1525.533223] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1537.691638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1537.793461] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1537.807594] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1537.848478] Lustre: lustre-MDT0000: new disk, initializing [ 1537.873652] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1537.881308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1539.355053] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1543.481298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1543.524758] Lustre: 125223:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1543.547915] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1543.549632] Lustre: Skipped 1 previous similar message [ 1543.582656] Lustre: lustre-MDT0001: new disk, initializing [ 1543.612976] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1543.626349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1543.630632] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1545.025074] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1548.697985] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1550.473514] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1552.656095] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1552.779352] Lustre: lustre-OST0000: new disk, initializing [ 1552.781095] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1552.783927] Lustre: 127192:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1552.813791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1553.015327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1553.018699] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1553.068162] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1554.824116] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1558.638895] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1560.103903] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1560.187910] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1561.662378] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1561.743246] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1563.618887] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1563.619172] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1563.620125] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1563.621435] LustreError: Skipped 1 previous similar message [ 1563.629169] Lustre: Skipped 1 previous similar message [ 1568.530806] Lustre: server umount lustre-OST0000 complete [ 1573.856703] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1573.857140] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1573.860351] Lustre: Skipped 1 previous similar message [ 1573.862077] Lustre: Skipped 1 previous similar message [ 1578.976416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1578.979094] Lustre: Skipped 2 previous similar messages [ 1584.095132] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1584.156873] Lustre: server umount lustre-MDT0000 complete [ 1585.472361] LustreError: 126018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610477 with bad export cookie 7191839697657392536 [ 1585.475937] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1585.476798] LustreError: 126018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1585.611777] Lustre: server umount lustre-MDT0001 complete [ 1588.545565] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:48:00 (1781610480) [ 1590.846692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1591.023210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1592.360864] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1595.255380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1596.743156] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1599.047415] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1600.906367] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1603.129195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1605.097570] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1607.314581] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1619.878182] Lustre: *** cfs_fail_loc=159, val=0*** [ 1619.880357] Lustre: Skipped 2 previous similar messages [ 1636.322174] Lustre: lustre-MDT0000: Client 23d8e590-3fb8-4e66-aed8-ccf6ceabde7c (at 192.168.202.13@tcp) reconnecting [ 1636.802854] Lustre: *** cfs_fail_loc=159, val=0*** [ 1636.804906] Lustre: Skipped 3 previous similar messages [ 1653.213863] Lustre: lustre-MDT0000: Client 23d8e590-3fb8-4e66-aed8-ccf6ceabde7c (at 192.168.202.13@tcp) reconnecting [ 1668.064562] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1668.065543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1668.067175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1668.067181] Lustre: Skipped 2 previous similar messages [ 1668.079723] Lustre: Skipped 1 previous similar message [ 1674.123552] Lustre: server umount lustre-OST0000 complete [ 1677.280083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1677.282363] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1681.493152] Lustre: server umount lustre-MDT0000 complete [ 1682.674171] LustreError: 129582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610575 with bad export cookie 7191839697657393404 [ 1682.676774] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1682.679747] LustreError: 129582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1682.795652] Lustre: server umount lustre-MDT0001 complete [ 1686.054396] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 1686.978287] Key type lgssc unregistered [ 1687.094302] LNet: 133403:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1687.096494] LNetError: 133403:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1687.104263] LNet: Removed LNI 192.168.202.113@tcp [ 1687.397520] Key type .llcrypt unregistered [ 1687.399098] Key type ._llcrypt unregistered [ 1696.213981] Key type ._llcrypt registered [ 1696.215733] Key type .llcrypt registered [ 1696.249352] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1696.696516] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1696.736637] alg: No test for adler32 (adler32-zlib) [ 1697.602314] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 1697.692715] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 1699.279252] Key type lgssc registered [ 1699.703966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1702.498458] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1702.509223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1703.682957] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1705.264654] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1708.308069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1708.435442] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1709.834037] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1712.187218] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1713.938945] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1716.164832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1716.292530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1718.259076] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1720.715734] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1728.613832] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1735.489177] Lustre: *** cfs_fail_loc=159, val=0*** [ 1751.007648] Lustre: lustre-MDT0000: Client f5e2426b-8faf-4409-9160-5019e0ee946d (at 192.168.202.13@tcp) reconnecting [ 1751.687902] Lustre: *** cfs_fail_loc=159, val=0*** [ 1767.906308] Lustre: lustre-MDT0001: Client f5e2426b-8faf-4409-9160-5019e0ee946d (at 192.168.202.13@tcp) reconnecting [ 1768.405549] Lustre: *** cfs_fail_loc=159, val=0*** [ 1768.407823] Lustre: Skipped 3 previous similar messages [ 1783.788157] Lustre: lustre-MDT0001: Client f5e2426b-8faf-4409-9160-5019e0ee946d (at 192.168.202.13@tcp) reconnecting [ 1784.986188] Lustre: 137846:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1785.783204] Lustre: *** cfs_fail_loc=159, val=0*** [ 1785.785259] Lustre: Skipped 4 previous similar messages [ 1802.208954] Lustre: lustre-MDT0000: Client cbaaee19-d78e-480f-8acb-e11a56d4d8ab (at 192.168.202.13@tcp) reconnecting [ 1802.774865] Lustre: *** cfs_fail_loc=159, val=0*** [ 1802.776367] Lustre: Skipped 13 previous similar messages [ 1819.092711] Lustre: lustre-MDT0000: Client cbaaee19-d78e-480f-8acb-e11a56d4d8ab (at 192.168.202.13@tcp) reconnecting [ 1819.629159] Lustre: 138137:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1821.153447] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1821.156188] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1821.156992] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1821.159654] Lustre: Skipped 1 previous similar message [ 1826.273579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1826.275848] Lustre: Skipped 2 previous similar messages [ 1826.965457] Lustre: server umount lustre-OST0000 complete [ 1831.394373] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1831.394378] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.396288] LustreError: Skipped 1 previous similar message [ 1831.400904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1831.403154] Lustre: Skipped 1 previous similar message [ 1834.524982] Lustre: server umount lustre-MDT0000 complete [ 1835.748417] LustreError: 137966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610728 with bad export cookie 5856145152508286331 [ 1835.751346] LustreError: 137966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1835.752221] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1835.882263] Lustre: server umount lustre-MDT0001 complete [ 1839.758592] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 1840.839076] Key type lgssc unregistered [ 1840.994573] LNet: 139187:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1840.999414] LNetError: 139187:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1841.008478] LNet: Removed LNI 192.168.202.113@tcp [ 1841.431551] Key type .llcrypt unregistered [ 1841.433647] Key type ._llcrypt unregistered [ 1850.495102] Key type ._llcrypt registered [ 1850.496172] Key type .llcrypt registered [ 1850.532252] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1850.957551] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1850.967201] alg: No test for adler32 (adler32-zlib) [ 1851.821365] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 1851.897618] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 1853.479185] Key type lgssc registered [ 1853.829448] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1856.320873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1856.326386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1857.479631] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1858.780879] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1861.685475] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1861.802022] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1863.058445] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1865.360593] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1867.178024] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1869.436968] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1869.548413] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1871.340301] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1871.591929] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1873.803965] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1881.833257] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 1886.313126] Lustre: 143185:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1886.954107] Lustre: 143233:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1887.418188] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 1888.532585] Lustre: 143380:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 1888.827184] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 1888.829635] Lustre: Skipped 1 previous similar message [ 1889.123781] Lustre: 143478:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1892.325694] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1892.326036] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1892.329968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1892.330889] LustreError: Skipped 1 previous similar message [ 1892.336588] Lustre: Skipped 1 previous similar message [ 1896.268951] Lustre: server umount lustre-OST0000 complete [ 1897.951821] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1897.953898] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1897.958412] Lustre: Skipped 1 previous similar message [ 1897.960576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1902.560147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1902.560595] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1903.644429] Lustre: server umount lustre-MDT0000 complete [ 1904.844098] LustreError: 140559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610797 with bad export cookie 16557686377439726325 [ 1904.847145] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1904.847929] LustreError: 140559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1904.954481] Lustre: server umount lustre-MDT0001 complete [ 1908.663829] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 1909.726911] Key type lgssc unregistered [ 1909.856391] LNet: 144534:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1909.860098] LNetError: 144534:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1909.866313] LNet: Removed LNI 192.168.202.113@tcp [ 1910.182201] Key type .llcrypt unregistered [ 1910.183165] Key type ._llcrypt unregistered [ 1919.557488] Key type ._llcrypt registered [ 1919.558516] Key type .llcrypt registered [ 1919.594893] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 1919.870133] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1919.882246] alg: No test for adler32 (adler32-zlib) [ 1920.739798] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 1920.826778] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 1922.407123] Key type lgssc registered [ 1922.760683] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1925.237305] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1925.242709] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1926.408512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.734666] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1930.522241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1930.645996] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1931.947137] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1934.241560] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1935.991664] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1938.391707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1938.531166] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1940.426827] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 1940.582593] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 1942.755856] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1950.826902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 1955.312918] Lustre: *** cfs_fail_loc=159, val=0*** [ 1970.658126] Lustre: lustre-MDT0000: Client e5cf9976-31ca-44e7-b71d-823b98c678f0 (at 192.168.202.13@tcp) reconnecting [ 1987.042342] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1987.045376] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1987.048373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1987.049566] Lustre: Skipped 1 previous similar message [ 1988.562146] Lustre: server umount lustre-OST0000 complete [ 1992.160817] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1992.161094] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1992.161830] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1992.161835] Lustre: Skipped 1 previous similar message [ 1992.163073] LustreError: Skipped 1 previous similar message [ 1992.167669] Lustre: Skipped 1 previous similar message [ 1995.937901] Lustre: server umount lustre-MDT0000 complete [ 1997.182335] LustreError: 145906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610889 with bad export cookie 3400985696374820520 [ 1997.185105] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1997.185871] LustreError: 145906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1997.310790] Lustre: server umount lustre-MDT0001 complete [ 2000.999127] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2001.952921] Key type lgssc unregistered [ 2002.072353] LNet: 149613:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2002.075439] LNetError: 149613:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2002.088305] LNet: Removed LNI 192.168.202.113@tcp [ 2002.386296] Key type .llcrypt unregistered [ 2002.387774] Key type ._llcrypt unregistered [ 2011.668570] Key type ._llcrypt registered [ 2011.669628] Key type .llcrypt registered [ 2011.707691] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2012.083219] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2012.092902] alg: No test for adler32 (adler32-zlib) [ 2012.967043] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2013.054349] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2014.639121] Key type lgssc registered [ 2014.992803] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2017.508152] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2017.515842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2018.673519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2019.926835] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2022.826298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2022.965483] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2024.334745] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2026.630479] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2028.306497] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2030.525459] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2030.672536] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2032.528754] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2032.744491] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2034.918848] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2044.004538] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2058.306456] Lustre: lustre-OST0000: evicting 97156e00-0830-4378-92b5-1d366cae661b (at 192.168.202.13@tcp) by administrative request [ 2058.310851] Lustre: Skipped 1 previous similar message [ 2084.320423] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2084.323464] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2084.328421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2085.346539] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2085.347204] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2085.349988] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2085.362697] Lustre: Skipped 1 previous similar message [ 2090.465459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2095.589180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2095.592271] Lustre: Skipped 2 previous similar messages [ 2097.119227] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2097.182779] Lustre: server umount lustre-OST0000 complete [ 2099.680482] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2099.683698] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2099.689589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2104.996804] Lustre: server umount lustre-MDT0000 complete [ 2105.824201] LustreError: 151003:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2105.829176] LustreError: 151003:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2106.544316] LustreError: 150985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781610999 with bad export cookie 787932414495085718 [ 2106.551285] LustreError: 150985:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2106.552265] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2106.708754] Lustre: server umount lustre-MDT0001 complete [ 2110.514672] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2111.568500] Key type lgssc unregistered [ 2111.703415] LNet: 155088:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2111.705895] LNetError: 155088:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2111.712453] LNet: Removed LNI 192.168.202.113@tcp [ 2112.029230] Key type .llcrypt unregistered [ 2112.030210] Key type ._llcrypt unregistered [ 2138.080151] Key type ._llcrypt registered [ 2138.081192] Key type .llcrypt registered [ 2138.116853] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2138.606493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2138.625370] alg: No test for adler32 (adler32-zlib) [ 2139.475503] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2139.561054] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2141.135196] Key type lgssc registered [ 2141.489042] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2144.083340] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2144.088605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2145.239163] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2146.607135] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2149.394175] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2149.536963] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2150.914915] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2153.257428] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2154.998114] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2157.231892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2157.362831] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2159.261180] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2160.421229] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2161.535419] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2169.638305] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2174.944629] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2174.945916] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2174.947697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2174.950472] Lustre: Skipped 1 previous similar message [ 2174.954095] Lustre: Skipped 1 previous similar message [ 2176.522862] Lustre: server umount lustre-OST0000 complete [ 2179.799744] Lustre: server umount lustre-MDT0000 complete [ 2180.066291] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2180.067109] LustreError: 157554:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2180.071456] Lustre: Skipped 1 previous similar message [ 2180.078571] LustreError: 157554:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2181.124647] LustreError: 159409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611073 with bad export cookie 11583946563087266639 [ 2181.128648] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2181.128902] LustreError: 159409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2181.253428] Lustre: server umount lustre-MDT0001 complete [ 2184.951092] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2185.996967] Key type lgssc unregistered [ 2186.122310] LNet: 161114:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2186.124475] LNetError: 161114:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2186.132275] LNet: Removed LNI 192.168.202.113@tcp [ 2186.415498] Key type .llcrypt unregistered [ 2186.416594] Key type ._llcrypt unregistered [ 2195.323038] Key type ._llcrypt registered [ 2195.323965] Key type .llcrypt registered [ 2195.360223] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2195.716417] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2195.758571] alg: No test for adler32 (adler32-zlib) [ 2196.623091] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2196.701222] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2198.279177] Key type lgssc registered [ 2198.619189] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2203.411068] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 07:58:15 (1781611095) [ 2208.611751] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 2211.764692] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2223.562208] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2223.567187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2224.654734] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2224.668629] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2224.705438] Lustre: lustre-MDT0000: new disk, initializing [ 2224.730688] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2224.740299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2226.134304] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2230.215022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2230.260282] Lustre: 165347:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2230.285933] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2230.288405] Lustre: Skipped 1 previous similar message [ 2230.325762] Lustre: lustre-MDT0001: new disk, initializing [ 2230.357759] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2230.368731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2230.371703] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2231.691174] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2235.219723] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2236.963103] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2239.138174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2239.236886] Lustre: lustre-OST0000: new disk, initializing [ 2239.238698] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2239.241512] Lustre: 167316:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2239.271550] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2241.138180] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2244.087188] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2244.091539] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2244.123035] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2244.739106] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2249.186637] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2249.186914] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2249.188848] LustreError: Skipped 1 previous similar message [ 2249.193151] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2249.195738] Lustre: Skipped 1 previous similar message [ 2253.965114] Lustre: server umount lustre-OST0000 complete [ 2255.839880] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2255.842555] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2255.846723] Lustre: Skipped 1 previous similar message [ 2255.848771] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2259.424165] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2259.424724] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2261.346905] Lustre: server umount lustre-MDT0000 complete [ 2262.605171] LustreError: 165339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611155 with bad export cookie 6774920146342346065 [ 2262.607738] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2262.610953] LustreError: 165339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2262.726731] Lustre: server umount lustre-MDT0001 complete [ 2266.494368] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2267.632766] Key type lgssc unregistered [ 2267.784440] LNet: 169219:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2267.788252] LNetError: 169219:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2267.799454] LNet: Removed LNI 192.168.202.113@tcp [ 2268.130594] Key type .llcrypt unregistered [ 2268.131818] Key type ._llcrypt unregistered [ 2277.696446] Key type ._llcrypt registered [ 2277.698457] Key type .llcrypt registered [ 2277.736668] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2278.131899] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2278.157736] alg: No test for adler32 (adler32-zlib) [ 2279.022873] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2279.126898] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2280.719190] Key type lgssc registered [ 2281.092949] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2283.707143] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2283.713787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2284.884400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2286.298161] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2289.139805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2289.270110] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2290.639858] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2293.036859] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2294.835263] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2297.134808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2297.260907] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2299.285591] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2301.686656] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2310.565908] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2314.452260] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2314.459760] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2314.464445] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2314.467559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2314.471086] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2314.806972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2315.147521] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2315.149988] Lustre: Skipped 1 previous similar message [ 2315.157812] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2315.163060] Lustre: Skipped 1 previous similar message [ 2315.164554] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2315.166585] Lustre: Skipped 1 previous similar message [ 2315.168645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2315.171775] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2315.174574] Lustre: Skipped 1 previous similar message [ 2315.497654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2315.839668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2316.157177] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2316.159560] Lustre: Skipped 2 previous similar messages [ 2316.165892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2316.170126] Lustre: Skipped 2 previous similar messages [ 2316.172320] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2316.174369] Lustre: Skipped 2 previous similar messages [ 2316.176310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2316.180155] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2316.183407] Lustre: Skipped 2 previous similar messages [ 2316.528852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2316.851643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2317.193816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2317.504071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2317.859149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2318.176209] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2318.178112] Lustre: Skipped 5 previous similar messages [ 2318.183718] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2318.187041] Lustre: Skipped 5 previous similar messages [ 2318.188455] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2318.191229] Lustre: Skipped 5 previous similar messages [ 2318.193188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2318.197696] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2318.199247] Lustre: Skipped 5 previous similar messages [ 2318.493472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2318.778290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2319.076442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2319.374102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2319.676647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2319.962949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2320.269387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2320.564157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2320.867715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2321.163214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2321.483812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2321.784190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2322.088662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2322.378152] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2322.381031] Lustre: Skipped 13 previous similar messages [ 2322.388171] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2322.393986] Lustre: Skipped 13 previous similar messages [ 2322.396356] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2322.399465] Lustre: Skipped 13 previous similar messages [ 2322.401305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2322.404679] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2322.406343] Lustre: Skipped 13 previous similar messages [ 2322.723420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2323.020311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2323.316335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2323.622429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2323.955337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2324.262176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2324.553559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2324.846953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2325.150118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2325.439694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2325.743967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2326.022294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2326.332328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2326.630240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2326.927353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2327.216833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2327.551643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2327.905121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2328.220128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2328.527431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2328.839700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2329.137855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2329.437156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2329.737535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2330.048431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2330.353184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2330.648803] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2330.650858] Lustre: Skipped 26 previous similar messages [ 2330.655834] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2330.659148] Lustre: Skipped 26 previous similar messages [ 2330.660571] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2330.662333] Lustre: Skipped 26 previous similar messages [ 2330.663880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2330.667100] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2330.668709] Lustre: Skipped 26 previous similar messages [ 2330.972944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.312520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.610297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2331.904570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.196339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.498996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2332.817054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2333.144380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2333.460017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2333.787148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.103345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.412452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2334.717521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.022892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.321748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.626990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2335.931855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.263453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.567849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2336.901730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2337.214210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2337.550928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2337.852013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.173382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.487956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.809734] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2338.813981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2338.814170] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2339.133750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2339.462856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2339.783124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.125250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.415647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2340.701982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.001689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.308913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.603188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2341.887448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.179470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.510989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2342.793348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2343.077397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2343.377705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2343.700790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.042049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.360377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2344.661395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.014276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.320509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.621662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2345.910974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.221783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.504993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.791307] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2346.793263] Lustre: Skipped 51 previous similar messages [ 2346.798339] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2346.801875] Lustre: Skipped 51 previous similar messages [ 2346.803346] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2346.805159] Lustre: Skipped 51 previous similar messages [ 2346.806535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2346.809693] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2346.811438] Lustre: Skipped 51 previous similar messages [ 2347.094641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2347.428096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2347.741173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.055851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.358264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2348.705505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.051598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.352653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.658071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2349.986374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.300703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.607242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2350.897765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.204338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.538584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2351.926338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2352.223226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2352.516879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2352.822607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.122058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.447693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2353.751371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.055795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.356668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.667483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2354.981016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.274270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.568768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2355.887568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.192283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.493637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2356.824825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.229945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.545291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2357.848157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2358.158180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2358.466976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2358.776422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.094295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.407299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2359.738661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.042224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.320244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.605361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2360.918576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.229032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.535957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2361.840454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.168057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.480693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2362.785504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.099664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.424789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2363.741873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.072088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.389166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2364.729932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.032235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.345420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.643619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2365.954569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.237162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.535380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2366.817912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.096188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.379975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2367.682212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.008273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.338191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.646955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2368.968079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.272171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.583400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2369.869534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.180678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.505012] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2370.823605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.131955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.459529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2371.784804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.109280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.443417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2372.756746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.058694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.361970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.660444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2373.972431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.305784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2374.644833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.026990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.375024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2375.773355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.165391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2376.577361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.012481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.383503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2377.707465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.027484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.424856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2378.749288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.085224] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2379.087751] Lustre: Skipped 100 previous similar messages [ 2379.093741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2379.096807] Lustre: Skipped 100 previous similar messages [ 2379.098287] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2379.100339] Lustre: Skipped 100 previous similar messages [ 2379.101903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.105223] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2379.106794] Lustre: Skipped 100 previous similar messages [ 2379.415523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2379.734122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.024186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.332859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.654995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2380.973862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.276820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.571817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2381.866624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.162216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.443913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.715664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2382.985024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.276324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.589778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2383.912902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.234979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.580280] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2384.584051] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2384.584983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2384.871465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.155327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.440403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2385.761616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.058488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.354725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.656775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2386.964825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.246229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.543774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2387.839829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.146913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.442989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2388.747651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.051370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.377850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.692604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2389.998827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.292830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.605671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2390.901894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.199059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.488445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2391.800251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.099926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.397117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2392.790592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.217286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2393.653442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.074269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.494903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2394.920569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.305210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2395.707394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.138602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.544657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2396.885977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.204221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.515724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2397.844046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.181904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.572374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2398.887051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.215834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.580842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2399.900861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.198469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.499502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2400.794325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.088458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.385300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.675280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2401.966225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.259739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.565119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2402.897275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.201629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.491159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2403.787157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.091156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.384522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.677345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2404.968777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.273402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.561538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2405.858692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.169064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.498294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2406.868640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.275298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.596532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2407.900916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.206233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.510785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2408.814097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.123601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.418867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2409.719816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.013611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.292426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.594444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2410.902740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.228536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.536493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2411.836981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.131378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.441079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2412.742304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.055692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.421201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2413.754973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.057810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.368188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2414.707778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.034339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.338664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.652448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2415.969745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.309379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2416.693862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.113139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.534091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2417.897939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.284963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2418.680178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.034049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.389140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.686035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2419.984708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.282158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.574919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2420.863384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.167206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.444068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2421.725359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.023949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.360954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.652226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2422.927914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.198511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.479775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2423.787450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.096335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.388073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.689923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2424.981677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.289095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.587703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2425.883792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.189675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.492657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2426.780585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.115464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.424668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2427.743175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.081081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.418222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2428.728601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.059294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.377363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2429.692288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.002373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.305687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.599490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2430.912236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.222017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.528603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2431.832071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.135075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.426453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2432.711595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.031203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.357116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.644028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2433.946155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.272803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.589941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2434.883793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.180045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.477159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2435.789676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.083955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.383128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2436.689258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.010399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.305741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.605890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2437.920942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.239264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.586432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2438.966460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.283587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.573135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2439.874374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.194291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.532326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2440.853508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.169495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.483308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2441.812924] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2441.817288] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2441.817346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.125395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.436945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2442.768828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.093169] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2443.095608] Lustre: Skipped 201 previous similar messages [ 2443.101931] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2443.105051] Lustre: Skipped 201 previous similar messages [ 2443.107496] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2443.109024] Lustre: Skipped 201 previous similar messages [ 2443.110570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.113688] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2443.115881] Lustre: Skipped 201 previous similar messages [ 2443.441398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2443.781795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.150482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.464201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2444.805325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.124481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.472330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2445.776223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.079024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.382482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.681548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2446.979100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.283989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.581618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2447.885192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.179989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.492190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2448.768236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.078622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.363615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.651175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2449.935343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.234401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.527722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2450.819023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.109364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.421480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.704585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2451.994509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.289829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.604352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2452.895027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.198817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.482509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2453.782949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.070447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.367419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.653692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2454.964206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.251338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.541886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2455.838609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.158940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.456972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2456.743685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.033429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.368903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.666726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2457.976979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.268388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.563585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2458.859113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.168778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.492607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2459.793419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.091698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.390576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.675516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2460.972761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.269891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.569209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2461.873468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.215633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.525887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2462.834623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.139094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.443078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2463.750069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.065144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.362549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.674562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2464.973228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.295152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.600675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2465.909870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.206151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.512433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2466.819268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.132252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.425169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2467.712714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.010276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.336429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.639271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2468.947369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.251673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.558818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2469.849332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.129508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.410687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2470.714919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.008940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.317554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.611505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.934058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.226743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.546317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.850374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.161878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.453607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.745387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.030430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.352144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.649397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.944386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.260286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.586327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.894556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.211295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.515345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.848550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.166733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.495644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.791509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.083809] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2478.089411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.089525] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2478.380931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.667253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.955196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.258341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.545850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.835101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.133492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.442604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.744642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.035645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.315193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.605508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.883889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.168865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.449959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.744512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.034584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.338564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.630575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.956794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.268057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.578504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.886160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.221109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.521223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.829149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.132247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.442782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.740307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.050349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.359163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.675424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.968297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.265730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.561348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.873150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.164443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.490364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.799276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.137657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.446925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.770682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.078155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.394487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.695260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.997743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.311443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.649837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.971869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.285946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.613314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.917640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.218458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.511879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.803160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.103714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.397911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.716825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.009879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.324659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.622166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.911271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.215348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.548184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.861689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.173479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.480030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.803507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.110795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.408194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.734117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.083398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.415512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.725377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.056089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.388862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.685967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.992990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.295991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.616104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.904546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.195188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.501358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.815815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.116811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.406961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.710620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.072709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.385725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.743759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.064356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.398635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.694631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.967918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.274834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.604916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.918506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.220073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.509302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.819495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.129077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.457668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.768852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.100023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.412243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.731808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.052980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.398203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.706174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.014225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.332865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.667504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.962068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.291953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.610308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.945595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.268394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.602614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.905610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.232218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.582597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.937644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.276684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.706510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.097314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.484962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.854739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.228493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.582405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.966425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.298282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.673755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.987986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.276914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.596388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.903580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.194349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.499146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.804948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.112803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.404526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.694452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.991475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.293377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.598973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.921565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.219881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.535822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.833528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.134186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.419399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.724328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.015031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.307080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.618611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.939566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.233074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.529363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.830147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.147568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.447401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.736762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.027393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.354087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.664484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.988629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.289330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.617669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.918876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.224095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.519308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.837622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.150705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.468301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.788541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.096152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.395847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.700578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.009919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.330258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.636546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.952493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.265213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.594426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.902307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.211655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.493703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.815371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.103017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.387392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.674226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.974807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.267311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.569966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.871676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.175694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.462168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.753995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.032505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.336494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.635743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.942629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.249670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.555906] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2541.559546] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2541.560150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.862169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.162063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.484320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.807675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.109300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.420695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.725845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.056210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.368061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.673331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.989504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.308452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.623244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.936595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.227451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.545526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.839258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.133454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.449671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.777772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.093234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.408420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.707480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.032463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.337224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.644366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.943055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.259422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.584433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.888723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.191469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.519394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.808399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.114767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.433832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.762054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.072826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.386767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.684695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.989153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.285463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.588580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.902906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.225304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.515202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.811912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.120064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.448498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.731351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.031728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.332833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.651924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.941337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.248771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.543432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.866219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.160545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.456309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.754556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.053878] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2560.057619] LustreError: 170630:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2560.059626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.358985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.678301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.018081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.376391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.681875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.993212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.287580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.588577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.894679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.195943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.518249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.850843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.162571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.488462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.845702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.174162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.530162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.884204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.201063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.535837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.848190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.168721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.469634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.794367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.103453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.412946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.710449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.028172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.320038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.616061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.908709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.206270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.503724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.820405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.112571] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2571.115627] Lustre: Skipped 413 previous similar messages [ 2571.121463] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2571.126481] Lustre: Skipped 413 previous similar messages [ 2571.128682] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2571.130556] Lustre: Skipped 413 previous similar messages [ 2571.132254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.136788] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2571.139333] Lustre: Skipped 413 previous similar messages [ 2571.432011] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.720195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.080584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.399421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.711547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.040130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.353403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.668286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.008526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.318519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.603257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.939664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.248083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.547068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.859382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.154483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.475994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.778597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.090963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.376953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.683510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.978433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.307146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.696048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.064710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.350804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.635518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.946051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.284471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.567443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.847316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.146491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.470400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.758763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.048750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.344250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.659264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.944611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.234558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.526377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.830138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.125355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.413023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.689074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.978502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.268366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.557310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.838127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.131766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.416112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.710623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.000851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.297714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.581488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.870092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.152284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.446695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.734319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.040675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.341025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.636712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.920516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.206785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.501791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.813771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.099441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.384296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.664873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.983686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.296049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.609178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.913558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.247958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.588064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.919472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.242883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.560607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.855157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.157937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.457586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.752720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.045597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.331919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.642452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.948650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.239328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.523168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.803978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.089952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.378589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.660579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.958058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.269202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.571594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.848328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.136171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.429482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.711888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.018945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.339974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.662836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.947037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.246162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.530121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.815438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.109470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.405871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.707762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.019706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.355438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.752585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.087978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.381990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.671456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.961581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.265897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.540573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.816165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.095337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.398268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.706278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.998052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.285575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.587336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.871950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.180756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.474857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.779770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.086488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.377382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.667575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.969592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.262694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.564359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.858401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.166980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.455076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.741789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.042061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.334962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.626227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.915673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.225075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.540539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.838563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.150764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.452363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.764553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.068053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.367253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.658918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.968947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.263657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.557672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.852018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.159956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.458795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.750815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.062352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.391091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.680320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.974435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.271346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.590236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.881355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.178393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.475181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.782465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.086096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.399771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.707143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.017395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.326223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.617958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.910392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.231236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.573745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.867483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.241255] LustreError: 172429:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2685.244197] LustreError: 170630:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 2749.919810] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2749.922887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2750.432310] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2750.435418] LustreError: Skipped 1 previous similar message [ 2750.436873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2752.560520] Lustre: server umount lustre-OST0000 complete [ 2755.552119] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2755.554656] LustreError: Skipped 1 previous similar message [ 2755.556367] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2755.558232] Lustre: Skipped 1 previous similar message [ 2761.184242] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2761.186139] Lustre: Skipped 3 previous similar messages [ 2766.304296] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2766.305968] Lustre: Skipped 1 previous similar message [ 2768.351231] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2768.487160] Lustre: server umount lustre-MDT0000 complete [ 2769.738922] LustreError: 170591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611662 with bad export cookie 14363064204182621417 [ 2769.743578] LustreError: 170591:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2769.743725] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2769.875136] Lustre: server umount lustre-MDT0001 complete [ 2773.813483] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2774.833692] Key type lgssc unregistered [ 2774.963355] LNet: 225647:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2774.967290] LNetError: 225647:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2774.976323] LNet: Removed LNI 192.168.202.113@tcp [ 2775.245470] Key type .llcrypt unregistered [ 2775.246504] Key type ._llcrypt unregistered [ 2784.253396] Key type ._llcrypt registered [ 2784.255060] Key type .llcrypt registered [ 2784.293488] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2784.646369] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2784.734559] alg: No test for adler32 (adler32-zlib) [ 2785.587330] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2785.673581] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2787.255134] Key type lgssc registered [ 2787.601447] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2790.145847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2790.152814] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2791.307579] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2792.607301] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2795.321058] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2795.447799] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2796.775958] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2799.119353] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2800.992691] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2803.887437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2804.039759] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2805.999355] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2808.447204] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2816.363022] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 2820.856173] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2821.466249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2821.473118] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2821.477450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.480927] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2821.480940] LustreError: 227058:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 2824.152722] Lustre: lustre-OST0000: Not available for connect from 192.168.202.13@tcp (stopping) [ 2825.696053] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2825.700412] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2825.707203] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2826.721649] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2826.722113] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2828.256046] Lustre: server umount lustre-OST0000 complete [ 2841.559112] LustreError: 227058:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 2856.308293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2856.432434] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2858.532241] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2860.932732] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2865.596473] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 2866.660281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.664270] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2867.374113] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fa9c5e0e000.ost_server_uuid 50 [ 2867.846282] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fa9c5e0e000.ost_server_uuid in FULL state after 0 sec [ 2871.777674] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2871.781741] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2871.783304] Lustre: Skipped 1 previous similar message [ 2874.955942] Lustre: server umount lustre-OST0000 complete [ 2876.896224] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2876.896665] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2876.900104] Lustre: Skipped 1 previous similar message [ 2876.904602] Lustre: Skipped 1 previous similar message [ 2882.326051] Lustre: server umount lustre-MDT0000 complete [ 2883.039864] LustreError: 227039:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2883.044410] LustreError: 227039:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2883.513135] LustreError: 231224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611776 with bad export cookie 8160854333156587080 [ 2883.516996] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2883.517463] LustreError: 231224:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2883.632548] Lustre: server umount lustre-MDT0001 complete [ 2887.008781] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 2888.012343] Key type lgssc unregistered [ 2888.137363] LNet: 232260:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2888.141118] LNetError: 232260:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2888.150298] LNet: Removed LNI 192.168.202.113@tcp [ 2888.427218] Key type .llcrypt unregistered [ 2888.428350] Key type ._llcrypt unregistered [ 2899.816108] Key type ._llcrypt registered [ 2899.817217] Key type .llcrypt registered [ 2899.858735] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2900.335284] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2900.342114] alg: No test for adler32 (adler32-zlib) [ 2901.191741] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 2901.272967] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 2902.855193] Key type lgssc registered [ 2903.229409] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2906.312354] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2906.320836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2907.482878] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2908.790838] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2911.745799] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2911.901777] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2913.208996] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2914.161842] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 2916.321426] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2916.439260] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2917.478891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 2918.425210] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2921.255216] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2921.326802] Lustre: lustre-OST0001: new disk, initializing [ 2921.329768] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2921.334102] Lustre: 236276:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2921.371593] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2921.885709] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2921.888669] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2921.917953] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2923.293281] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2928.864717] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 2933.456659] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 2942.944046] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2942.947476] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2942.953461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2943.456474] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2943.458062] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2943.462327] Lustre: Skipped 1 previous similar message [ 2945.835368] Lustre: server umount lustre-MDT0000 complete [ 2947.047453] LustreError: 235098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611839 with bad export cookie 7173643860028408014 [ 2947.051710] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2947.171656] Lustre: server umount lustre-MDT0001 complete [ 2957.761752] Lustre: server umount lustre-OST0000 complete [ 2969.246665] Lustre: server umount lustre-OST0001 complete [ 2976.144977] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 2979.772647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2979.922682] LustreError: 239375:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2979.929050] LustreError: 239375:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 2979.945139] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2981.329944] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2984.110898] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2985.654910] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2986.637085] Lustre: 240519:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 2988.813193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2988.935427] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2988.938311] Lustre: Skipped 1 previous similar message [ 2989.986451] LustreError: 241061:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2989.994989] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 2990.890817] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2993.661048] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2995.698530] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 2998.849387] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3000.063321] Lustre: 242385:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3004.896259] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3004.898644] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3004.901895] Lustre: Skipped 1 previous similar message [ 3004.903727] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3004.905316] Lustre: Skipped 3 previous similar messages [ 3010.524138] Lustre: server umount mylustre-MDT0000 complete [ 3010.532921] LustreError: 239376:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3010.539616] LustreError: 239376:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3011.818575] LustreError: 239352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611904 with bad export cookie 7173643860028410779 [ 3011.822449] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3011.824054] LustreError: 239352:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3011.950326] Lustre: server umount mylustre-MDT0001 complete [ 3022.856515] Lustre: server umount mylustre-OST0000 complete [ 3034.348494] Lustre: server umount mylustre-OST0001 complete [ 3041.227365] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3045.000111] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3045.147941] LustreError: 244689:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3045.170670] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3045.172618] Lustre: Skipped 1 previous similar message [ 3046.458579] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3049.269634] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3050.726144] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3051.670447] Lustre: 245831:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3053.849103] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3055.012994] LustreError: 246182:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3055.016399] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 3055.900807] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3058.708240] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3060.697820] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3063.800660] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3064.936868] Lustre: 247698:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3069.919888] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3069.922397] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3069.927710] Lustre: Skipped 3 previous similar messages [ 3069.929874] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3069.931665] Lustre: Skipped 4 previous similar messages [ 3072.921846] Lustre: server umount tfs-MDT0000 complete [ 3074.130062] LustreError: 244665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781611966 with bad export cookie 7173643860028412445 [ 3074.133824] LustreError: 244665:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3074.133832] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3074.254166] Lustre: server umount tfs-MDT0001 complete [ 3085.765619] Lustre: server umount tfs-OST0000 complete [ 3097.322213] Lustre: server umount tfs-OST0001 complete [ 3104.056034] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3107.299291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3107.466098] LustreError: 249945:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3107.487347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3107.489399] Lustre: Skipped 3 previous similar messages [ 3108.792961] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3111.706929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3113.132363] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3114.076095] Lustre: 251088:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3116.297666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3117.478313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 3118.252653] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3121.172086] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3123.321338] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3126.418763] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3127.674296] Lustre: 252956:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3129.759103] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:13:41 (1781612021) [ 3142.624362] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3142.626571] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3142.630318] Lustre: Skipped 3 previous similar messages [ 3142.631934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3142.633830] Lustre: Skipped 4 previous similar messages [ 3147.677260] Lustre: server umount lustre-MDT0000 complete [ 3148.258627] LustreError: 249946:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3148.262662] LustreError: 249946:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3148.931133] LustreError: 249920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612041 with bad export cookie 7173643860028413866 [ 3148.934341] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3148.936895] LustreError: 249920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3153.376971] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3153.377983] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3153.380571] Lustre: Skipped 4 previous similar messages [ 3153.382032] Lustre: Skipped 4 previous similar messages [ 3155.132124] Lustre: server umount lustre-MDT0001 complete [ 3162.494852] Lustre: server umount lustre-OST0000 complete [ 3169.881622] Lustre: server umount lustre-OST0001 complete [ 3171.999402] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 3175.054980] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3187.517308] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3187.517308] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3187.517308] [ 3187.527668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3187.615927] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3187.629410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3187.668616] Lustre: lustre-MDT0000: new disk, initializing [ 3187.699505] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3187.701500] Lustre: Skipped 3 previous similar messages [ 3187.707078] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3189.099900] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3193.212222] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3193.212222] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3193.212222] [ 3193.220452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3193.265376] Lustre: 256131:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3193.289899] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3193.292210] Lustre: Skipped 1 previous similar message [ 3193.329361] Lustre: lustre-MDT0001: new disk, initializing [ 3193.371619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3193.374924] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3194.653544] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3198.748873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3198.849709] Lustre: lustre-OST0000: new disk, initializing [ 3198.851415] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3198.853910] Lustre: 257716:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3200.724816] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3200.760845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3200.765887] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3200.802960] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3204.758761] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3204.822416] Lustre: lustre-OST0001: new disk, initializing [ 3204.824192] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3204.826767] Lustre: 258585:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3206.857865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3206.862197] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3206.902400] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3206.908556] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3211.184374] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 08:15:03 (1781612103) [ 3214.577583] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 08:15:06 (1781612106) [ 3226.592400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3226.592426] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3226.594890] LustreError: Skipped 1 previous similar message [ 3226.598863] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3226.601772] Lustre: Skipped 2 previous similar messages [ 3232.063842] Lustre: server umount lustre-OST0000 complete [ 3232.740731] LustreError: 257701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3232.746076] LustreError: 257701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3239.519324] Lustre: server umount lustre-MDT0000 complete [ 3240.746269] LustreError: 256124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612133 with bad export cookie 7173643860028415196 [ 3240.748915] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3240.749670] LustreError: 256124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3255.263158] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3255.351643] Lustre: server umount lustre-MDT0001 complete [ 3258.873455] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 3268.565328] Lustre: server umount lustre-OST0001 complete [ 3270.841809] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 3277.135573] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3288.833766] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3288.924808] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3288.936363] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3288.974956] Lustre: lustre-MDT0000: new disk, initializing [ 3289.004529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3290.323242] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3294.471092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3294.534328] Lustre: 263121:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3294.684418] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3296.031916] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3300.000495] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3302.357859] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3304.780192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3304.926449] Lustre: 265089:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3306.240647] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3306.243879] Lustre: Skipped 1 previous similar message [ 3306.246077] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3306.283769] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3306.870220] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3310.590312] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3316.706787] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3316.707186] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3316.709592] LustreError: Skipped 1 previous similar message [ 3316.713929] Lustre: Skipped 5 previous similar messages [ 3316.716254] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3316.718930] Lustre: Skipped 10 previous similar messages [ 3319.009239] Lustre: server umount lustre-OST0000 complete [ 3326.427855] Lustre: server umount lustre-MDT0000 complete [ 3326.944571] LustreError: 263134:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3326.950501] LustreError: 263134:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 3327.753668] LustreError: 263113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612220 with bad export cookie 7173643860028416666 [ 3327.757215] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3327.757732] LustreError: 263113:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3327.905926] Lustre: server umount lustre-MDT0001 complete [ 3331.492708] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 3332.470498] Key type lgssc unregistered [ 3332.599479] LNet: 266846:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3332.603224] LNetError: 266846:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3332.614323] LNet: Removed LNI 192.168.202.113@tcp [ 3332.916104] Key type .llcrypt unregistered [ 3332.917267] Key type ._llcrypt unregistered [ 3343.005343] Key type ._llcrypt registered [ 3343.006258] Key type .llcrypt registered [ 3343.050921] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3343.460119] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3343.505971] alg: No test for adler32 (adler32-zlib) [ 3344.352397] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 3344.432435] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 3346.007112] Key type lgssc registered [ 3346.355065] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3348.855258] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3348.861618] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3350.023190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3351.487297] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3354.384822] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3354.510731] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3355.837251] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3358.179801] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3359.998741] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3362.188654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3362.298767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3364.172319] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3366.616067] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3366.935875] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3370.466124] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3370.466440] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3370.471189] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3373.644590] Lustre: server umount lustre-OST0000 complete [ 3375.071955] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3375.075727] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3375.080237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3379.680552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3379.681164] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3381.081474] Lustre: server umount lustre-MDT0000 complete [ 3382.383272] LustreError: 268266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612274 with bad export cookie 8174281721345598062 [ 3382.386349] LustreError: 268266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3382.386977] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3382.515196] Lustre: server umount lustre-MDT0001 complete [ 3386.369270] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 3387.495490] Key type lgssc unregistered [ 3387.628472] LNet: 271866:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3387.630880] LNetError: 271866:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3387.639408] LNet: Removed LNI 192.168.202.113@tcp [ 3387.985898] Key type .llcrypt unregistered [ 3387.987810] Key type ._llcrypt unregistered [ 3394.915237] Key type ._llcrypt registered [ 3394.916433] Key type .llcrypt registered [ 3394.960949] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3395.501538] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3395.520616] alg: No test for adler32 (adler32-zlib) [ 3396.391838] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 3396.493086] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 3398.087257] Key type lgssc registered [ 3398.521365] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3401.598815] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3401.606413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3417.055911] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3422.175445] LustreError: 273037:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.113@tcp: failed processing log, type 4: rc = -110 [ 3448.799186] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3451.198065] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3453.884331] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3454.700267] Lustre: server umount lustre-OST0000 complete [ 3460.707365] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 3461.723276] Key type lgssc unregistered [ 3461.844378] LNet: 274665:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3461.847228] LNetError: 274665:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3461.854314] LNet: Removed LNI 192.168.202.113@tcp [ 3462.142595] Key type .llcrypt unregistered [ 3462.143949] Key type ._llcrypt unregistered [ 3475.611632] Key type ._llcrypt registered [ 3475.613085] Key type .llcrypt registered [ 3475.661172] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3475.984241] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3476.020309] alg: No test for adler32 (adler32-zlib) [ 3476.882592] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 3476.977271] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 3478.559171] Key type lgssc registered [ 3478.919593] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3483.496940] loop0: detected capacity change from 0 to 204800000 [ 3483.600525] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3483.604920] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3483.736296] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.122413] loop0: detected capacity change from 0 to 204800000 [ 3484.180329] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.184877] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.322190] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.703470] loop0: detected capacity change from 0 to 204800000 [ 3484.767038] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.770305] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3484.816145] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3485.207032] loop0: detected capacity change from 0 to 204800000 [ 3485.269985] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3485.727855] loop0: detected capacity change from 0 to 204800000 [ 3485.739535] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3487.179390] loop0: detected capacity change from 0 to 204800000 [ 3487.187433] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3488.522605] loop0: detected capacity change from 0 to 204800000 [ 3488.535135] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3489.913182] loop0: detected capacity change from 0 to 204800000 [ 3489.929406] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3491.279050] loop0: detected capacity change from 0 to 204800000 [ 3491.307256] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3491.316750] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3491.353688] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3491.357471] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3491.362332] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3491.367070] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3491.372682] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3491.375994] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3491.382326] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3491.387796] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3491.391983] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3491.397111] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3491.400833] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3491.404756] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3491.412282] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3491.416788] Lustre: 278052:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3494.041252] Lustre: server umount MGS complete [ 3494.344949] loop0: detected capacity change from 0 to 204800000 [ 3494.367157] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3494.533421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3494.549965] Lustre: lustre-MDT0000: Aborting client recovery [ 3494.850806] loop1: detected capacity change from 0 to 204800000 [ 3494.880247] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3494.921967] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3494.925794] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3494.929526] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3494.937014] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3494.940933] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3494.944557] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3494.947906] Lustre: 278482:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3494.955119] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3495.051401] Lustre: lustre-MDT0001: Aborting client recovery [ 3495.346572] loop2: detected capacity change from 0 to 204800000 [ 3495.378400] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3495.416057] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3495.419953] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3495.423915] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3495.428165] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3495.433617] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3495.439129] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3495.444584] Lustre: 278563:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3495.494087] Lustre: Found index 0 for lustre-OST0000, updating log [ 3495.507952] Lustre: 278581:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3495.529623] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3495.532586] Lustre: Skipped 1 previous similar message [ 3495.539418] Lustre: lustre-OST0000: Aborting recovery [ 3495.836871] loop3: detected capacity change from 0 to 204800000 [ 3495.873320] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3495.925094] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3495.930080] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3495.935340] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3495.939713] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3495.943710] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3495.947370] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3495.950255] Lustre: 278647:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3501.032521] LustreError: 278426:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 3505.122654] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3505.122816] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3505.128639] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3505.130886] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3505.132240] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3505.138771] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3506.629833] LustreError: 278393:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 238us: rc = -2 [ 3520.480529] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3520.481273] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3520.494176] Lustre: Skipped 3 previous similar messages [ 3520.496133] Lustre: Skipped 2 previous similar messages [ 3523.935814] Lustre: server umount lustre-MDT0000 complete [ 3524.224919] LustreError: 278378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612416 with bad export cookie 2652676440752680037 [ 3524.226328] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3524.230429] LustreError: 278378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3525.599614] LustreError: 278392:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3525.599797] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3525.601044] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3525.601057] Lustre: Skipped 1 previous similar message [ 3525.607069] LustreError: 278392:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3525.611524] Lustre: Skipped 1 previous similar message [ 3529.631498] LustreError: 278390:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3529.631670] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3529.637569] LustreError: 278390:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3529.640248] Lustre: Skipped 2 previous similar messages [ 3530.486118] Lustre: server umount lustre-MDT0001 complete [ 3545.055141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3545.097789] Lustre: server umount lustre-OST0000 complete [ 3551.523827] Lustre: server umount lustre-OST0001 complete [ 3554.386310] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 08:20:46 (1781612446) [ 3563.074699] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 3565.958434] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3577.403237] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3577.516734] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3577.530351] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3577.572134] Lustre: lustre-MDT0000: new disk, initializing [ 3577.599274] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3577.601432] Lustre: Skipped 1 previous similar message [ 3577.607847] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3579.012691] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3583.267042] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.310697] Lustre: 282715:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3583.337025] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3583.338830] Lustre: Skipped 1 previous similar message [ 3583.376464] Lustre: lustre-MDT0001: new disk, initializing [ 3583.413983] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3583.428245] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3583.432544] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3584.840943] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3588.711573] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3590.429348] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3592.620686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3592.722962] Lustre: lustre-OST0000: new disk, initializing [ 3592.724647] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3592.728853] Lustre: 284684:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3592.732240] Lustre: 284684:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 3592.761091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3594.642604] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3598.123496] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3600.381236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3600.384069] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3600.420915] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3605.533553] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3617.093686] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3625.702135] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3631.271549] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3682.272670] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3682.275241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3682.276210] LustreError: Skipped 1 previous similar message [ 3682.280420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3682.283061] Lustre: Skipped 1 previous similar message [ 3684.624851] Lustre: server umount lustre-OST0000 complete [ 3687.392611] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3687.393542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3687.397414] Lustre: Skipped 2 previous similar messages [ 3692.130170] Lustre: server umount lustre-MDT0000 complete [ 3692.512265] LustreError: 282727:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3692.520854] LustreError: 282727:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3693.407494] LustreError: 282708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612585 with bad export cookie 2652676440752682739 [ 3693.409802] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3693.412861] LustreError: 282708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3693.552876] Lustre: server umount lustre-MDT0001 complete [ 3696.476546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3697.811727] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3699.454617] Lustre: server umount MGS complete [ 3703.338040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3703.496256] LustreError: 288928:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3703.503350] LustreError: 288941:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3703.527975] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3704.872606] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3707.837338] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3709.377124] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3711.920334] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3713.643099] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3715.902311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3717.944388] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3720.459542] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3749.857598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3749.858974] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3749.859129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3749.859134] Lustre: Skipped 1 previous similar message [ 3749.860669] Lustre: Skipped 1 previous similar message [ 3755.729837] Lustre: server umount lustre-OST0000 complete [ 3759.071798] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3759.074026] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3770.336181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3770.338143] Lustre: Skipped 9 previous similar messages [ 3771.359100] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3771.424648] Lustre: server umount lustre-MDT0000 complete [ 3772.647052] LustreError: 288911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612665 with bad export cookie 2652676440752684580 [ 3772.650125] LustreError: 288911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3772.650366] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3772.776123] Lustre: server umount lustre-MDT0001 complete [ 3776.400286] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 3777.487137] Key type lgssc unregistered [ 3777.614407] LNet: 292889:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3777.616429] LNetError: 292889:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3777.624381] LNet: Removed LNI 192.168.202.113@tcp [ 3777.953206] Key type .llcrypt unregistered [ 3777.954368] Key type ._llcrypt unregistered [ 3789.924853] Key type ._llcrypt registered [ 3789.926510] Key type .llcrypt registered [ 3789.964464] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 3796.368234] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 3796.740756] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3796.787942] alg: No test for adler32 (adler32-zlib) [ 3797.655169] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 3797.744955] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 3799.327106] Key type lgssc registered [ 3799.682963] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3811.308676] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3811.317543] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3812.441294] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3812.460912] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3812.514577] Lustre: lustre-MDT0000: new disk, initializing [ 3812.541214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3812.548758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3814.048770] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3818.277741] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3818.321930] Lustre: 296817:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3818.359222] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3818.360884] Lustre: Skipped 1 previous similar message [ 3818.399962] Lustre: lustre-MDT0001: new disk, initializing [ 3818.431697] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3818.443675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3818.447729] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3819.769830] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3823.447114] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3825.272373] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3827.552039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3827.653899] Lustre: lustre-OST0000: new disk, initializing [ 3827.655512] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3827.657709] Lustre: 298785:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3827.684870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3829.764740] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3832.830256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3832.834643] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3832.873095] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3833.467465] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3836.826623] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 3843.090365] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 3852.817024] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 3859.367640] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 3904.482646] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3904.483878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3904.485436] LustreError: Skipped 1 previous similar message [ 3904.489717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3904.492326] Lustre: Skipped 1 previous similar message [ 3906.460371] Lustre: server umount lustre-OST0000 complete [ 3909.600942] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3909.601696] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3909.604134] Lustre: Skipped 2 previous similar messages [ 3909.608769] Lustre: Skipped 1 previous similar message [ 3914.720428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3914.723127] Lustre: Skipped 1 previous similar message [ 3919.840289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3919.843503] Lustre: Skipped 1 previous similar message [ 3921.887140] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3921.959688] Lustre: server umount lustre-MDT0000 complete [ 3923.436956] LustreError: 297613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612815 with bad export cookie 16029952921585046724 [ 3923.440132] LustreError: 297613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3923.441433] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3923.574875] Lustre: server umount lustre-MDT0001 complete [ 3926.827986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3928.209532] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3929.815386] Lustre: server umount MGS complete [ 3933.761191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3933.925399] LustreError: 302890:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3933.931166] LustreError: 302890:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3933.931871] LustreError: 302903:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 3933.959146] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3935.427696] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3938.574665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3938.709509] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3940.408908] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3943.539699] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3945.566148] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3948.048233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3948.177880] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3950.240404] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 3952.598204] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3985.379617] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3985.381186] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3985.381993] LustreError: Skipped 1 previous similar message [ 3985.388728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3985.390717] Lustre: Skipped 1 previous similar message [ 3990.303036] Lustre: server umount lustre-OST0000 complete [ 3995.103965] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3995.106038] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3995.109106] Lustre: Skipped 1 previous similar message [ 3995.110529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3997.799608] Lustre: server umount lustre-MDT0000 complete [ 3999.145536] LustreError: 302870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781612891 with bad export cookie 16029952921585048572 [ 3999.149278] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3999.150254] LustreError: 302870:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3999.287627] Lustre: server umount lustre-MDT0001 complete [ 4003.247799] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 4004.672398] Key type lgssc unregistered [ 4004.818463] LNet: 306856:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4004.821454] LNetError: 306856:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4004.835645] LNet: Removed LNI 192.168.202.113@tcp [ 4005.232476] Key type .llcrypt unregistered [ 4005.233810] Key type ._llcrypt unregistered [ 4012.002419] Key type ._llcrypt registered [ 4012.003440] Key type .llcrypt registered [ 4012.044944] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4017.714665] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4018.108154] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4018.171283] alg: No test for adler32 (adler32-zlib) [ 4019.049472] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 4019.145116] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 4020.735172] Key type lgssc registered [ 4021.102522] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4029.274428] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4035.280946] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4035.289304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4036.399307] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4036.409933] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4036.446047] Lustre: lustre-MDT0000: new disk, initializing [ 4036.469457] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4036.475982] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4037.767414] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4042.085971] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4042.190216] Lustre: lustre-OST0000: new disk, initializing [ 4042.191860] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4042.193631] Lustre: Skipped 1 previous similar message [ 4042.195544] Lustre: 310880:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4042.221216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4044.233911] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4044.674900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4044.677321] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4044.685699] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4232.574015] hrtimer: interrupt took 2027012 ns [ 4242.469645] Lustre: server umount lustre-MDT0000 complete [ 4245.683717] LustreError: 310063:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613138 with bad export cookie 17817790385844935028 [ 4245.692741] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4256.305365] Lustre: server umount lustre-OST0000 complete [ 4268.827176] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4272.384104] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4285.736862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4285.827918] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4285.840360] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4285.876626] Lustre: lustre-MDT0000: new disk, initializing [ 4285.904672] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4285.913840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4287.459474] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4292.097965] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4292.153706] Lustre: 315108:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4292.178924] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4292.181010] Lustre: Skipped 1 previous similar message [ 4292.218630] Lustre: lustre-MDT0001: new disk, initializing [ 4292.254541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4292.275239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4292.281889] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4293.932747] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4298.362746] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4300.556902] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4303.201608] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4303.331893] Lustre: lustre-OST0000: new disk, initializing [ 4303.335818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4303.339358] Lustre: 317075:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4303.385168] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4305.478274] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4308.478700] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4308.482479] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4308.520794] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4309.350400] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4310.932901] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4311.004334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4312.453465] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4312.537627] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4313.568849] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4313.569028] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4313.575272] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4313.577623] LustreError: Skipped 1 previous similar message [ 4315.362335] Lustre: server umount lustre-OST0000 complete [ 4318.175949] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4318.178073] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4318.180950] Lustre: Skipped 1 previous similar message [ 4318.182475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4318.184219] Lustre: Skipped 1 previous similar message [ 4322.969222] Lustre: server umount lustre-MDT0000 complete [ 4324.397437] LustreError: 315100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613216 with bad export cookie 17817790385847875805 [ 4324.403669] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4324.521742] Lustre: server umount lustre-MDT0001 complete [ 4325.744266] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 4326.352618] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 08:33:38 (1781613218) [ 4328.761239] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4328.905798] LustreError: 319386:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4328.912706] LustreError: 319386:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4328.928670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4330.307115] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4333.696671] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4335.342831] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4338.020714] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4340.029351] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4342.609589] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4344.951809] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4347.855714] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4350.758770] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4350.826098] Lustre: lustre-OST0001: new disk, initializing [ 4350.828631] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4350.832307] Lustre: 322217:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4350.871049] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4350.873754] Lustre: Skipped 2 previous similar messages [ 4353.117811] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4357.131683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4357.136657] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4357.152991] LustreError: 322956:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4357.446809] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4357.537613] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4358.182351] LustreError: 322956:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4358.187786] LustreError: 322956:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4359.202263] LustreError: 322956:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 4359.206537] LustreError: 322956:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 4359.738755] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fa9c5823000.ost_server_uuid 50 [ 4360.404670] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fa9c5823000.ost_server_uuid in FULL state after 0 sec [ 4361.251538] LustreError: 322961:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 4361.257125] LustreError: 322961:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 4362.652247] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9fa9c5823000.ost_server_uuid 50 [ 4363.237634] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9fa9c5823000.ost_server_uuid in FULL state after 0 sec [ 4364.330936] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4376.549173] Lustre: lustre-OST0001: Not available for connect from 192.168.202.13@tcp (stopping) [ 4376.553052] Lustre: Skipped 1 previous similar message [ 4377.574486] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.578850] Lustre: Skipped 1 previous similar message [ 4381.253446] Lustre: server umount lustre-OST0001 complete [ 4381.654499] LustreError: 321370:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.691642] LustreError: 321205:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.698020] LustreError: 321205:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4384.736344] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4384.737896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4384.738868] LustreError: Skipped 1 previous similar message [ 4384.739476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4384.744420] Lustre: Skipped 2 previous similar messages [ 4384.751331] Lustre: Skipped 3 previous similar messages [ 4385.698587] Lustre: server umount lustre-OST0000 complete [ 4389.337146] Lustre: server umount lustre-MDT0000 complete [ 4390.851069] LustreError: 321227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613283 with bad export cookie 17817790385847876673 [ 4390.853816] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4390.855690] LustreError: 321227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4390.996428] Lustre: server umount lustre-MDT0001 complete [ 4395.249306] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 4396.414227] Key type lgssc unregistered [ 4396.558427] LNet: 324901:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4396.562693] LNetError: 324901:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4396.571130] LNet: Removed LNI 192.168.202.113@tcp [ 4396.928217] Key type .llcrypt unregistered [ 4396.929605] Key type ._llcrypt unregistered [ 4407.822910] Key type ._llcrypt registered [ 4407.824159] Key type .llcrypt registered [ 4407.878652] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4408.242901] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4408.274032] alg: No test for adler32 (adler32-zlib) [ 4409.157800] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 4409.256569] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 4410.847169] Key type lgssc registered [ 4411.345347] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4414.683444] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4414.690412] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4415.928522] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4417.714667] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4420.538220] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4422.924612] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4423.080341] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4424.498680] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4428.007759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4428.127694] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4430.266251] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4432.881114] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4433.253185] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4433.262348] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4435.422575] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4435.509377] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4437.744766] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4440.462475] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 4441.577324] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 4441.577644] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 4447.782736] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 4447.874494] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4532.703938] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4532.707067] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4532.711746] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4533.731225] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 4533.736642] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4533.740122] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4537.402463] Lustre: server umount lustre-OST0001 complete [ 4538.852178] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4538.858719] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4540.895938] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4540.899464] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4540.903679] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4540.905975] Lustre: Skipped 1 previous similar message [ 4543.969619] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4543.970402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4543.970900] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4543.972192] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4543.976509] LustreError: 327974:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4543.978679] Lustre: Skipped 1 previous similar message [ 4545.230874] Lustre: server umount lustre-OST0000 complete [ 4549.088741] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4549.089585] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4549.094142] Lustre: Skipped 1 previous similar message [ 4552.738350] Lustre: server umount lustre-MDT0000 complete [ 4554.054589] LustreError: 327227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613446 with bad export cookie 6231747940085005757 [ 4554.059056] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4554.059449] LustreError: 327227:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4554.198396] Lustre: server umount lustre-MDT0001 complete [ 4558.013464] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 4559.065240] Key type lgssc unregistered [ 4559.202576] LNet: 331233:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4559.205887] LNetError: 331233:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4559.213417] LNet: Removed LNI 192.168.202.113@tcp [ 4559.506457] Key type .llcrypt unregistered [ 4559.507576] Key type ._llcrypt unregistered [ 4578.796306] Key type ._llcrypt registered [ 4578.797896] Key type .llcrypt registered [ 4578.841277] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4579.243820] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4579.261116] alg: No test for adler32 (adler32-zlib) [ 4580.135746] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 4580.232310] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 4581.831172] Key type lgssc registered [ 4582.290895] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4585.776491] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4585.783294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4586.963290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4588.364683] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4591.197932] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4591.340866] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4592.753708] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4593.759112] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 4595.864420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4595.984316] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4597.027597] LustreError: 335001:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4597.036242] LustreError: 335001:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4597.038755] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 4598.164239] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4601.123139] LustreError: 335003:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4601.135606] LustreError: 335003:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4601.202685] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4601.288902] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4603.426845] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4606.442743] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 4607.905654] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4611.913259] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 4616.676557] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4616.677693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4616.681649] Lustre: Skipped 3 previous similar messages [ 4616.684756] Lustre: Skipped 2 previous similar messages [ 4621.793886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4621.796243] Lustre: Skipped 2 previous similar messages [ 4622.046038] Lustre: server umount lustre-MDT0000 complete [ 4623.366329] LustreError: 336517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613515 with bad export cookie 15495797173943223367 [ 4623.369830] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4623.371165] LustreError: 336517:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4623.497272] Lustre: server umount lustre-MDT0001 complete [ 4635.145372] Lustre: server umount lustre-OST0000 complete [ 4645.722590] Lustre: server umount lustre-OST0001 complete [ 4651.130877] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4654.677185] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4654.812757] LustreError: 338771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4654.833153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4656.177951] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4659.153172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4660.812135] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4661.989795] Lustre: 339912:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4664.554047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4664.707857] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4664.710743] Lustre: Skipped 1 previous similar message [ 4665.767509] LustreError: 340268:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4665.777469] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 4667.034936] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4670.031599] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4672.317725] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4675.561830] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 4675.888309] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4677.253762] Lustre: 341780:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4679.647878] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4679.650446] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4679.656426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4680.676342] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4680.680526] Lustre: Skipped 2 previous similar messages [ 4684.502834] Lustre: server umount lustre-MDT0000 complete [ 4685.743038] LustreError: 339915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613578 with bad export cookie 15495797173943225026 [ 4685.746567] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4685.747433] LustreError: 339915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4685.792362] LustreError: 338767:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4685.793342] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4685.793846] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4685.793851] Lustre: Skipped 4 previous similar messages [ 4685.797572] LustreError: 338767:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4685.802521] Lustre: Skipped 1 previous similar message [ 4685.865919] Lustre: server umount lustre-MDT0001 complete [ 4693.372829] Lustre: server umount lustre-OST0000 complete [ 4694.742536] Lustre: server umount lustre-OST0001 complete [ 4696.138734] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 4696.719599] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 4697.372917] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 08:39:49 (1781613589) [ 4705.323035] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4708.719938] loop0: detected capacity change from 0 to 1099511627776 [ 4708.869254] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 4708.872080] *** VALIDATE xfs *** [ 4708.876502] XFS (loop0): Mounting V5 Filesystem [ 4708.886000] XFS (loop0): Ending clean mount [ 4709.347300] loop1: detected capacity change from 0 to 18691697672192 [ 4729.985621] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 4733.440610] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4745.375730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4745.492226] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4745.508964] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4745.543229] Lustre: lustre-MDT0000: new disk, initializing [ 4745.572977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4745.576141] Lustre: Skipped 1 previous similar message [ 4745.584173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4747.093411] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4751.416298] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.467031] Lustre: 346887:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4751.495944] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4751.497745] Lustre: Skipped 1 previous similar message [ 4751.535910] Lustre: lustre-MDT0001: new disk, initializing [ 4751.585037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4751.589111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4753.061830] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4756.966099] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4759.018099] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4761.539453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4761.650944] Lustre: lustre-OST0000: new disk, initializing [ 4761.653372] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4761.657322] Lustre: 348854:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4763.326123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4763.330069] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4763.370979] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4763.824559] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4767.645104] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4769.084632] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4769.159598] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4770.478727] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4770.550918] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4773.859068] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4773.859311] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4773.865609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4773.868527] Lustre: Skipped 2 previous similar messages [ 4777.300371] Lustre: server umount lustre-OST0000 complete [ 4784.096096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4784.098736] Lustre: Skipped 3 previous similar messages [ 4784.734578] Lustre: server umount lustre-MDT0000 complete [ 4786.065114] LustreError: 346878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613678 with bad export cookie 15495797173943226559 [ 4786.068274] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4786.070119] LustreError: 346878:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4786.187865] Lustre: server umount lustre-MDT0001 complete [ 4789.005486] XFS (loop0): Unmounting Filesystem [ 4790.826028] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 08:41:22 (1781613682) [ 4793.424895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4793.587433] LustreError: 351407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4793.618071] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4793.621423] Lustre: Skipped 2 previous similar messages [ 4795.238555] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4798.429637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4800.167213] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4802.700505] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4804.496467] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4806.895074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4809.091049] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4811.488416] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4829.665941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4829.668312] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4829.670889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4829.671844] Lustre: Skipped 4 previous similar messages [ 4829.674825] LustreError: Skipped 2 previous similar messages [ 4831.443698] Lustre: server umount lustre-OST0000 complete [ 4834.272077] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4838.995214] Lustre: server umount lustre-MDT0000 complete [ 4839.903945] LustreError: 352148:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4839.909549] LustreError: 352148:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4840.212243] LustreError: 353250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781613732 with bad export cookie 15495797173943227441 [ 4840.215515] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4840.216780] LustreError: 353250:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4840.333298] Lustre: server umount lustre-MDT0001 complete [ 4843.935488] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 4845.012723] Key type lgssc unregistered [ 4845.147533] LNet: 355054:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4845.150108] LNetError: 355054:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4845.159372] LNet: Removed LNI 192.168.202.113@tcp [ 4845.487136] Key type .llcrypt unregistered [ 4845.488434] Key type ._llcrypt unregistered [ 4856.240386] Key type ._llcrypt registered [ 4856.241600] Key type .llcrypt registered [ 4856.283524] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 4856.684526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4856.721073] alg: No test for adler32 (adler32-zlib) [ 4857.568330] Lustre: Lustre: Build Version: 2.17.54_40_g537638b [ 4857.648773] LNet: Added LNI 192.168.202.113@tcp [8/256/0/180] [ 4859.231094] Key type lgssc registered [ 4859.570891] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4862.074621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4862.081588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4863.241591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4864.555404] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4867.283177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4867.410482] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4868.865577] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4871.470205] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4873.487517] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4875.798995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4875.908840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4877.741880] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4880.162440] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4893.151846] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 4893.154506] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.159345] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4893.666203] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4893.670085] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4895.704908] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.13@tcp (stopping) [ 4898.787114] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4898.789047] Lustre: Skipped 1 previous similar message [ 4899.215170] Lustre: server umount lustre-MDT0001 complete [ 4900.820816] LustreError: 356595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4900.828056] LustreError: 356595:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4902.932555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4902.972644] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 4902.974803] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4902.993491] Lustre: 356586:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4903.125500] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4904.564304] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 4908.517407] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 4910.053646] LustreError: 360328:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 4910.057245] LustreError: 360328:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 4910.061373] LustreError: 356591:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 4910.077398] LustreError: 360331:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.113@tcp: cfg command failed: rc = -17 [ 4910.080339] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 4910.080339] [ 5208.024334] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.13@tcp (stopping) [ 5210.593257] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5210.593895] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5210.597584] Lustre: Skipped 1 previous similar message [ 5218.258287] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.13@tcp (stopping) [ 5218.261949] Lustre: Skipped 5 previous similar messages [ 5221.343125] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5221.401636] Lustre: server umount lustre-MDT0001 complete [ 5223.378416] LustreError: 356595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5224.886689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5224.922159] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5224.925068] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5224.941252] Lustre: 356588:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5225.053840] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5226.481459] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5230.564332] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5233.638946] LustreError: 372955:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5233.641630] LustreError: 372955:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5233.644988] LustreError: 356591:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5233.647566] LustreError: 356591:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5233.667765] LustreError: 372958:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 5233.671055] LustreError: 372958:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 5233.673412] LustreError: 372958:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.113@tcp: cfg command failed: rc = -17 [ 5233.676502] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.113@tcp [ 5233.676502] [ 5530.584473] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.13@tcp (stopping) [ 5530.586373] Lustre: Skipped 2 previous similar messages [ 5532.641950] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5532.642204] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5543.391106] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5543.446782] Lustre: server umount lustre-MDT0001 complete [ 5545.937963] LustreError: 357989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.13@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5546.877607] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5546.923505] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5546.925693] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5546.941892] Lustre: 356589:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5547.075519] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 5548.461567] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5552.104515] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5556.197534] LustreError: 385585:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5556.199868] LustreError: 385585:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5556.202528] LustreError: 356591:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5556.205275] LustreError: 356591:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 5556.224547] LustreError: 385588:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 5556.230326] LustreError: 385588:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.113@tcp: cfg command failed: rc = -114 [ 5556.233483] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 5556.233483] [ 5864.417595] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5864.418801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5864.421949] Lustre: Skipped 3 previous similar messages [ 5864.424690] Lustre: Skipped 9 previous similar messages [ 5864.428703] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5867.989524] Lustre: server umount lustre-MDT0000 complete [ 5869.161174] LustreError: 358439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781614761 with bad export cookie 10328514502512233121 [ 5869.164417] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5869.165023] LustreError: 358439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5869.535869] LustreError: 358604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5874.655688] LustreError: 357989:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5875.304730] Lustre: server umount lustre-MDT0001 complete [ 5876.596448] Lustre: server umount lustre-OST0000 complete [ 5879.464442] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 5882.705854] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 5893.805231] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:59:45 (1781614785) [ 5895.819719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5895.900655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5895.911254] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5895.946575] Lustre: lustre-MDT0000: new disk, initializing [ 5895.971173] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5895.979227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5897.227351] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5901.170235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5901.211735] Lustre: 400091:0:(mgs_llog.c:1447:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5901.235400] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5901.237593] Lustre: Skipped 1 previous similar message [ 5901.273621] Lustre: lustre-MDT0001: new disk, initializing [ 5901.305570] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5901.316388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5901.318838] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5902.574853] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5906.298543] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5908.067657] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5910.298225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5910.403158] Lustre: lustre-OST0000: new disk, initializing [ 5910.404758] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5910.406959] Lustre: 402058:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5910.440441] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5912.530278] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_default_debug -1 all [ 5913.271673] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5913.275519] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5913.310728] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5916.277167] Lustre: DEBUG MARKER: oleg213-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5921.759802] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5921.762620] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5921.766510] Lustre: Skipped 1 previous similar message [ 5924.945814] Lustre: server umount lustre-MDT0000 complete [ 5926.035122] LustreError: 400081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781614818 with bad export cookie 10328514502512235060 [ 5926.038529] LustreError: MGC192.168.202.113@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5926.038584] LustreError: 400898:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 5926.039582] LustreError: 400081:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5926.142647] Lustre: server umount lustre-MDT0001 complete [ 5937.563390] Lustre: server umount lustre-OST0000 complete [ 5946.999955] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing set_hostid [ 5950.209273] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing load_modules_local [ 5960.258739] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 5851 sec ========== 09:00:52 (1781614852) [ 5960.772872] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 09:00:52 (1781614852) === [ 5961.901175] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 09:00:53 (1781614853) === [ 5974.795862] Lustre: DEBUG MARKER: oleg213-server.virtnet: executing unload_modules_local [ 5975.887364] Key type lgssc unregistered [ 5976.020589] LNet: 408420:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5976.023738] LNetError: 408420:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5976.033390] LNet: Removed LNI 192.168.202.113@tcp [ 5976.345946] Key type .llcrypt unregistered [ 5976.347074] Key type ._llcrypt unregistered