[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 489782721 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003060] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008623] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010016] pid_max: default: 32768 minimum: 301 [ 0.011133] LSM: Security Framework initializing [ 0.012042] Yama: becoming mindful. [ 0.013040] SELinux: Initializing. [ 0.014074] *** VALIDATE selinux *** [ 0.022299] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027597] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028178] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029120] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030126] *** VALIDATE tmpfs *** [ 0.032245] *** VALIDATE proc *** [ 0.033258] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.036222] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037155] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039031] Spectre V2 : User space: Vulnerable [ 0.040010] Speculative Store Bypass: Vulnerable [ 0.043556] debug: unmapping init [mem 0xffffffff95459000-0xffffffff95460fff] [ 0.045231] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046730] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047029] ... version: 2 [ 0.048016] ... bit width: 48 [ 0.049016] ... generic registers: 4 [ 0.050017] ... value mask: 0000ffffffffffff [ 0.051015] ... max period: 00007fffffffffff [ 0.052015] ... fixed-purpose events: 3 [ 0.053017] ... event mask: 000000070000000f [ 0.054337] rcu: Hierarchical SRCU implementation. [ 0.056580] smp: Bringing up secondary CPUs ... [ 0.057684] x86: Booting SMP configuration: [ 0.058034] .... node #0, CPUs: #1 #2 #3 [ 0.061486] smp: Brought up 1 node, 4 CPUs [ 0.063017] smpboot: Max logical packages: 1 [ 0.064023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.155153] node 0 deferred pages initialised in 86ms [ 0.157141] devtmpfs: initialized [ 0.159242] x86/mm: Memory block size: 128MB [ 0.161914] gcov: version magic: 0x41383552 [ 0.165125] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.168095] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.171301] pinctrl core: initialized pinctrl subsystem [ 0.173182] [ 0.173691] ************************************************************* [ 0.176018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.179015] ** ** [ 0.181015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.184019] ** ** [ 0.186011] ** This means that this kernel is built to expose internal ** [ 0.188015] ** IOMMU data structures, which may compromise security on ** [ 0.191015] ** your system. ** [ 0.193012] ** ** [ 0.196013] ** If you see this message and you are not debugging the ** [ 0.198012] ** kernel, report this immediately to your vendor! ** [ 0.201015] ** ** [ 0.203013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.206015] ************************************************************* [ 0.208665] NET: Registered protocol family 16 [ 0.211440] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.214059] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.216062] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.219029] cpuidle: using governor menu [ 0.220769] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.223877] PCI: Using configuration type 1 for base access [ 0.226165] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.236159] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.239032] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.243036] cryptd: max_cpu_qlen set to 1000 [ 0.246260] ACPI: Added _OSI(Module Device) [ 0.247012] ACPI: Added _OSI(Processor Device) [ 0.249018] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.250051] ACPI: Added _OSI(Processor Aggregator Device) [ 0.256055] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.263566] ACPI: Interpreter enabled [ 0.265069] ACPI: PM: (supports S0 S3 S4 S5) [ 0.266014] ACPI: Using IOAPIC for interrupt routing [ 0.268111] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.271448] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.284913] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.287067] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.290028] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.292067] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.299368] acpiphp: Slot [2] registered [ 0.302178] acpiphp: Slot [5] registered [ 0.304366] acpiphp: Slot [6] registered [ 0.307153] acpiphp: Slot [7] registered [ 0.309184] acpiphp: Slot [8] registered [ 0.310239] acpiphp: Slot [9] registered [ 0.312210] acpiphp: Slot [10] registered [ 0.313202] acpiphp: Slot [3] registered [ 0.315171] acpiphp: Slot [4] registered [ 0.317149] acpiphp: Slot [11] registered [ 0.318157] acpiphp: Slot [12] registered [ 0.320144] acpiphp: Slot [13] registered [ 0.322224] acpiphp: Slot [14] registered [ 0.324168] acpiphp: Slot [15] registered [ 0.325163] acpiphp: Slot [16] registered [ 0.327176] acpiphp: Slot [17] registered [ 0.330149] acpiphp: Slot [18] registered [ 0.331102] acpiphp: Slot [19] registered [ 0.333118] acpiphp: Slot [20] registered [ 0.334125] acpiphp: Slot [21] registered [ 0.336114] acpiphp: Slot [22] registered [ 0.338140] acpiphp: Slot [23] registered [ 0.339108] acpiphp: Slot [24] registered [ 0.341100] acpiphp: Slot [25] registered [ 0.342136] acpiphp: Slot [26] registered [ 0.343121] acpiphp: Slot [27] registered [ 0.345131] acpiphp: Slot [28] registered [ 0.346146] acpiphp: Slot [29] registered [ 0.348105] acpiphp: Slot [30] registered [ 0.350591] acpiphp: Slot [31] registered [ 0.352097] PCI host bridge to bus 0000:00 [ 0.354032] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.356039] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.359039] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.362045] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.365039] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.368055] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.370316] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.373351] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.377501] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.388024] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.393018] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.396025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.398027] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.400022] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.403389] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.405815] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.409041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.411633] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.418014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.429021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.433789] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.439972] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.452027] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.460024] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.488027] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.499822] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.513021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.524019] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.557026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.568017] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.574030] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.581021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.599035] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.608947] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.620026] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.633028] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.651029] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.663031] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.672018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.677018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.692019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.703086] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.712026] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.717025] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.736027] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.748114] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.751498] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.753409] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.759275] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.763338] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.770284] iommu: Default domain type: Passthrough [ 0.772580] SCSI subsystem initialized [ 0.774224] ACPI: bus type USB registered [ 0.775141] usbcore: registered new interface driver usbfs [ 0.777156] usbcore: registered new interface driver hub [ 0.779135] usbcore: registered new device driver usb [ 0.781387] pps_core: LinuxPPS API ver. 1 registered [ 0.783086] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.787095] PTP clock support registered [ 0.789124] EDAC MC: Ver: 3.0.0 [ 0.791129] PCI: Using ACPI for IRQ routing [ 0.794020] NetLabel: Initializing [ 0.795014] NetLabel: domain hash size = 128 [ 0.797017] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.799122] NetLabel: unlabeled traffic allowed by default [ 0.802143] vgaarb: loaded [ 0.803331] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.805024] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.811409] clocksource: Switched to clocksource kvm-clock [ 0.935245] VFS: Disk quotas dquot_6.6.0 [ 0.936994] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.939589] *** VALIDATE ramfs *** [ 0.940861] *** VALIDATE hugetlbfs *** [ 0.942642] pnp: PnP ACPI init [ 0.945406] pnp: PnP ACPI: found 6 devices [ 0.963913] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.967619] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.969926] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.972374] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.975330] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.978226] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.981773] NET: Registered protocol family 2 [ 0.984668] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.991086] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.995054] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.000333] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.005056] TCP: Hash tables configured (established 65536 bind 65536) [ 1.008718] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.012628] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.016064] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.018848] NET: Registered protocol family 1 [ 1.022565] RPC: Registered named UNIX socket transport module. [ 1.025086] RPC: Registered udp transport module. [ 1.027015] RPC: Registered tcp transport module. [ 1.028922] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.031361] NET: Registered protocol family 44 [ 1.033139] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.035852] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.038025] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.040337] PCI: CLS 0 bytes, default 64 [ 1.041956] Unpacking initramfs... [ 2.421848] debug: unmapping init [mem 0xffff9824bcc54000-0xffff9824bffbffff] [ 2.426097] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.428451] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.431476] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.945528] Initialise system trusted keyrings [ 2.947254] Key type blacklist registered [ 2.949287] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.959240] zbud: loaded [ 2.962431] *** VALIDATE nfs *** [ 2.963731] *** VALIDATE nfs4 *** [ 2.965472] pstore: using deflate compression [ 2.969286] Platform Keyring initialized [ 3.091045] NET: Registered protocol family 38 [ 3.092388] Key type asymmetric registered [ 3.093192] Asymmetric key parser 'x509' registered [ 3.094319] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.096439] io scheduler mq-deadline registered [ 3.098390] io scheduler kyber registered [ 3.101323] io scheduler bfq registered [ 3.103533] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.107681] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.111560] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.115147] ACPI: Power Button [PWRF] [ 3.121498] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.128447] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.138928] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.143559] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.155403] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.181868] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.212694] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.217768] Non-volatile memory driver v1.3 [ 3.219273] Linux agpgart interface v0.103 [ 3.251067] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.254080] vda: detected capacity change from 0 to 74698752 [ 3.277329] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.280762] vdb: detected capacity change from 0 to 1073741824 [ 3.302174] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.305475] vdc: detected capacity change from 0 to 2621440000 [ 3.321579] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.325018] vdd: detected capacity change from 0 to 2621440000 [ 3.348455] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.352350] vde: detected capacity change from 0 to 4294967296 [ 3.370638] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.373446] vdf: detected capacity change from 0 to 4294967296 [ 3.383215] libphy: Fixed MDIO Bus: probed [ 3.405748] usbcore: registered new interface driver usbserial_generic [ 3.407437] usbserial: USB Serial support registered for generic [ 3.409311] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.413432] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.415299] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.417513] mousedev: PS/2 mouse device common for all mice [ 3.419605] rtc_cmos 00:05: RTC can wake from S4 [ 3.422063] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.422210] rtc_cmos 00:05: registered as rtc0 [ 3.426829] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.429382] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.430690] intel_pstate: CPU model not supported [ 3.436921] hid: raw HID events driver (C) Jiri Kosina [ 3.439267] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.439456] usbcore: registered new interface driver usbhid [ 3.444658] usbhid: USB HID core driver [ 3.446176] drop_monitor: Initializing network drop monitor service [ 3.448621] Initializing XFRM netlink socket [ 3.450705] NET: Registered protocol family 10 [ 3.467253] Segment Routing with IPv6 [ 3.469278] NET: Registered protocol family 17 [ 3.471886] mpls_gso: MPLS GSO support [ 3.479687] RAS: Correctable Errors collector initialized. [ 3.481919] AVX version of gcm_enc/dec engaged. [ 3.483965] AES CTR mode by8 optimization enabled [ 3.565092] sched_clock: Marking stable (3565067006, 0)->(4455265590, -890198584) [ 3.568214] registered taskstats version 1 [ 3.569904] Loading compiled-in X.509 certificates [ 3.572517] zswap: loaded using pool lzo/zbud [ 3.604057] Key type big_key registered [ 3.619526] Key type encrypted registered [ 3.620832] ima: No TPM chip found, activating TPM-bypass! [ 3.623176] ima: Allocated hash algorithm: sha1 [ 3.624945] ima: No architecture policies found [ 3.626792] evm: Initialising EVM extended attributes: [ 3.628762] evm: security.selinux [ 3.629974] evm: security.ima [ 3.631149] evm: security.capability [ 3.632644] evm: HMAC attrs: 0x1 [ 3.635103] rtc_cmos 00:05: setting system clock to 2026-08-08 12:05:47 UTC (1786190747) [ 3.641688] debug: unmapping init [mem 0xffffffff96403000-0xffffffff965fffff] [ 3.645111] debug: unmapping init [mem 0xffffffff95182000-0xffffffff95458fff] [ 3.658145] Write protecting the kernel read-only data: 28672k [ 3.661669] debug: unmapping init [mem 0xffffffff93803000-0xffffffff939fffff] [ 3.664679] debug: unmapping init [mem 0xffffffff94114000-0xffffffff941fffff] [ 3.705690] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.713642] systemd[1]: Detected virtualization kvm. [ 3.714843] systemd[1]: Detected architecture x86-64. [ 3.716418] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.743372] systemd[1]: No hostname configured. [ 3.744822] systemd[1]: Set hostname to . [ 3.746273] random: systemd: uninitialized urandom read (16 bytes read) [ 3.748040] systemd[1]: Initializing machine ID from random generator. [ 3.882693] random: systemd: uninitialized urandom read (16 bytes read) [ 3.885229] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.889465] random: systemd: uninitialized urandom read (16 bytes read) [ 3.891589] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.897472] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Timers. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Apply Kernel Variables... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Sockets. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.511294] device-mapper: uevent: version 1.0.3 [ 4.513742] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.261758] random: fast init done [ 5.278479] virtio_net virtio0 ens2: renamed from eth0 [ 5.322926] scsi host0: ata_piix [ 5.346260] scsi host1: ata_piix [ 5.348072] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.350729] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.107087] dracut-initqueue[582]: RTNETLINK answers: File exists [ 10.138461] random: crng init done [ 10.140040] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.604752] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.845690] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.120444] SELinux: Disabled at runtime. [ 12.181664] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.190501] systemd[1]: Detected virtualization kvm. [ 12.192336] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.699264] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.703086] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.710231] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.714921] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.718617] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.726640] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.735435] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice User and Session Slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File Syste[ 12.815195] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS m... [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.232076] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.613823] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.630245] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.755073] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.773668] EDAC sbridge: Ver: 1.1.2 [ 15.655394] Key type dns_resolver registered [ 15.979613] NFS: Registering the id_resolver key type [ 15.981735] Key type id_resolver registered [ 15.983516] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg233-server login: [ 41.209421] libcfs: loading out-of-tree module taints kernel. [ 41.231284] Key type ._llcrypt registered [ 41.233085] Key type .llcrypt registered [ 41.286597] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 55.919450] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 57.726660] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 57.740561] alg: No test for adler32 (adler32-zlib) [ 59.335368] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 60.020636] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 61.913065] Key type lgssc registered [ 64.132894] Lustre: Echo OBD driver; http://www.lustre.org/ [ 86.803511] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 93.271009] hrtimer: interrupt took 154812 ns [ 137.792783] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 152.245487] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 152.320784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 153.688857] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 153.740325] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 153.829834] Lustre: lustre-MDT0000: new disk, initializing [ 153.940268] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 153.958141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 159.354665] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 176.712234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 176.936775] Lustre: 6512:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 177.007374] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 177.017096] Lustre: Skipped 1 previous similar message [ 177.113475] Lustre: lustre-MDT0001: new disk, initializing [ 177.180477] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 177.211367] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 177.226734] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 182.847739] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 188.718566] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 202.549253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 202.916587] Lustre: lustre-OST0000: new disk, initializing [ 202.920450] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 202.937205] Lustre: 8455:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 203.065680] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 210.532879] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 213.009046] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 213.022210] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 213.070722] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 228.200050] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 228.493463] Lustre: lustre-OST0001: new disk, initializing [ 228.499140] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 228.509750] Lustre: 9527:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 228.649714] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 235.543568] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 236.635689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 236.649914] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 236.754213] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 248.413967] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 260.814614] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 267.501736] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing check_logdir /tmp/testlogs/ [ 274.534151] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing yml_node [ 279.463710] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 282.486592] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 285.461077] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 287.094130] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 8 08:10:29 EDT 2026 [ 305.377274] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 307.695549] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 324.064777] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.065501] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 324.076968] Lustre: Skipped 1 previous similar message [ 324.085546] Lustre: Skipped 3 previous similar messages [ 329.006971] Lustre: server umount lustre-MDT0000 complete [ 333.378555] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191077 with bad export cookie 3708153902010055985 [ 333.391266] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 333.397068] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 334.304206] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 334.305311] LustreError: 6518:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 334.305863] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 334.321558] Lustre: Skipped 3 previous similar messages [ 334.348447] LustreError: 6518:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 338.340819] LustreError: 8311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 338.340871] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 338.382968] Lustre: Skipped 2 previous similar messages [ 339.816914] Lustre: server umount lustre-MDT0001 complete [ 343.726451] Lustre: server umount lustre-OST0000 complete [ 347.398725] Lustre: server umount lustre-OST0001 complete [ 353.744745] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 363.437644] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 396.894987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 397.045630] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 397.078367] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 397.124783] Lustre: lustre-MDT0000: new disk, initializing [ 397.180886] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 397.196335] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 401.576768] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 412.327966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 412.488160] Lustre: 15079:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 412.544860] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 412.548466] Lustre: Skipped 1 previous similar message [ 412.610645] Lustre: lustre-MDT0001: new disk, initializing [ 412.713763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 412.749758] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 412.758629] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 416.880266] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 426.769554] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 433.308610] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 440.295779] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 440.725739] Lustre: lustre-OST0000: new disk, initializing [ 440.729447] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 440.737583] Lustre: 17048:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 440.841786] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 442.707737] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 442.720116] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 442.845874] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 447.045374] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 456.893567] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 462.671435] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 462.920904] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 467.333732] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 467.610830] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 473.580565] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 473.587539] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 473.613608] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 473.622606] Lustre: Skipped 1 previous similar message [ 475.700498] Lustre: server umount lustre-OST0000 complete [ 483.811527] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 483.819722] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 483.827855] Lustre: Skipped 2 previous similar messages [ 483.841782] Lustre: Skipped 1 previous similar message [ 485.923123] Lustre: server umount lustre-MDT0000 complete [ 488.685446] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191232 with bad export cookie 3708153902010057518 [ 488.687391] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 488.694172] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 489.003806] Lustre: server umount lustre-MDT0001 complete [ 493.034305] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:13:55 (1786191235) [ 500.959568] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 501.405171] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.423779] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 501.508293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 505.177477] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 506.848541] LustreError: 19314:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.969896] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 512.876706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 516.669043] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 523.334681] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 528.462655] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 534.396408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 539.513581] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 546.423672] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 549.587360] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 557.541917] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 557.546460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 557.553213] Lustre: Skipped 2 previous similar messages [ 557.564653] Lustre: Skipped 2 previous similar messages [ 561.099479] Lustre: server umount lustre-MDT0000 complete [ 562.662404] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 562.679683] LustreError: 19313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 564.340623] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191308 with bad export cookie 3708153902010058400 [ 564.342486] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 564.347403] LustreError: 19294:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 564.668878] Lustre: server umount lustre-MDT0001 complete [ 577.460482] Lustre: server umount lustre-OST0000 complete [ 595.388246] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 603.867978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 604.178271] LustreError: 23691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 604.249441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 604.256383] Lustre: Skipped 2 previous similar messages [ 607.710585] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 615.301340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 619.375295] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 621.918736] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 627.665746] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.727298] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 641.135651] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 641.284944] Lustre: lustre-OST0001: new disk, initializing [ 641.288219] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 641.293423] Lustre: 26006:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 642.965547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 642.976504] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 643.045061] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 646.513935] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 654.390682] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 657.156450] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 667.105043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 667.111490] LustreError: Skipped 1 previous similar message [ 667.114603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 667.121403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 671.079570] Lustre: server umount lustre-MDT0000 complete [ 673.766227] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 673.789054] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 673.816782] LustreError: 23672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191417 with bad export cookie 3708153902010059247 [ 673.823196] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 673.824043] LustreError: 23672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 674.210571] Lustre: server umount lustre-MDT0001 complete [ 687.170568] Lustre: server umount lustre-OST0000 complete [ 700.761429] Lustre: server umount lustre-OST0001 complete [ 712.952913] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 720.161683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 723.450685] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 729.658805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 733.185108] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 735.346805] Lustre: 30172:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 740.573609] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 740.826578] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 740.829775] Lustre: Skipped 5 previous similar messages [ 745.515742] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 745.961483] LustreError: 30523:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 745.986310] LustreError: 30523:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 752.314522] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 757.325057] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 762.723197] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 765.109797] Lustre: 32054:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 770.017981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 770.023113] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 770.025822] Lustre: Skipped 5 previous similar messages [ 770.037936] Lustre: Skipped 7 previous similar messages [ 774.154903] Lustre: server umount lustre-MDT0000 complete [ 776.830757] LustreError: 29004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191520 with bad export cookie 3708153902010060542 [ 776.840066] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 776.846359] LustreError: 29004:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 777.065787] Lustre: server umount lustre-MDT0001 complete [ 789.561294] Lustre: server umount lustre-OST0000 complete [ 802.165481] Lustre: server umount lustre-OST0001 complete [ 807.912517] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:19:10 (1786191550) [ 825.191381] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 831.250253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 833.804583] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 839.145660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 842.337426] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 844.250718] Lustre: 36001:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 849.180792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 854.401251] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 861.155221] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 866.138714] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 871.767738] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 874.383379] Lustre: 37877:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 880.616261] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 880.620961] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 880.625921] Lustre: Skipped 1 previous similar message [ 880.630719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 884.028206] Lustre: server umount lustre-MDT0000 complete [ 886.388865] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191630 with bad export cookie 3708153902010061648 [ 886.396688] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 886.399686] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 886.581187] Lustre: server umount lustre-MDT0001 complete [ 898.295058] Lustre: server umount lustre-OST0000 complete [ 911.196913] Lustre: server umount lustre-OST0001 complete [ 917.562576] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:21:00 (1786191660) [ 927.990410] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 936.781956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 937.003685] LustreError: 40019:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 937.016800] LustreError: 40019:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 939.677956] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 944.801805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 947.304681] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 949.229436] Lustre: 41168:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 953.241547] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 957.701938] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 963.362731] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 968.275479] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 974.371622] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 977.835280] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 977.839473] Lustre: Skipped 1 previous similar message [ 980.960366] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 980.966067] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 980.976395] Lustre: Skipped 3 previous similar messages [ 986.146309] Lustre: server umount lustre-MDT0000 complete [ 988.813228] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191732 with bad export cookie 3708153902010062747 [ 988.816274] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 988.819584] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 995.010271] Lustre: server umount lustre-MDT0001 complete [ 1001.321691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1013.730455] LustreError: 3639:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff982504354380 x1872956807820672/t0(0) o250->MGC192.168.202.133@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1014.055543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1014.059574] Lustre: Skipped 9 previous similar messages [ 1016.495475] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1021.612649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1023.978488] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1024.662350] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1029.474717] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1030.491943] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1033.858508] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1034.927410] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1039.334340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1039.337440] Lustre: Skipped 19 previous similar messages [ 1044.094907] Lustre: server umount lustre-MDT0000 complete [ 1046.195541] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191790 with bad export cookie 3708153902010063881 [ 1046.198976] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1046.203896] LustreError: 39999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1046.499900] Lustre: server umount lustre-MDT0001 complete [ 1059.865243] Lustre: server umount lustre-OST0000 complete [ 1072.533308] Lustre: server umount lustre-OST0001 complete [ 1077.425274] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:23:40 (1786191820) [ 1087.239314] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1093.868765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1096.921556] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1102.646881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1105.947312] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1107.708219] Lustre: 48800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1107.714149] Lustre: 48800:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 1112.112209] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1116.063362] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1121.474460] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1125.393264] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1130.561721] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1133.590280] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1142.244275] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1142.250559] Lustre: Skipped 12 previous similar messages [ 1147.428189] Lustre: server umount lustre-MDT0000 complete [ 1150.079133] LustreError: 47631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191893 with bad export cookie 3708153902010064679 [ 1150.084680] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1150.094162] LustreError: 47631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1150.267444] Lustre: server umount lustre-MDT0001 complete [ 1163.082761] Lustre: server umount lustre-OST0000 complete [ 1175.902626] Lustre: server umount lustre-OST0001 complete [ 1181.895286] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 08:25:25 (1786191925) [ 1183.503598] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1184.559093] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 08:25:27 (1786191927) [ 1191.352992] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 1197.445057] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1212.171048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1212.314913] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1212.371709] Lustre: lustre-MDT0000: new disk, initializing [ 1212.439211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1214.616888] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1220.656750] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1225.189642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1225.413112] Lustre: 54866:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1227.516580] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1227.537627] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1229.684470] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1235.652262] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1236.939476] Lustre: DEBUG MARKER: create test files [ 1257.047168] Lustre: server umount lustre-OST0000 complete [ 1269.338859] Lustre: server umount lustre-MDT0000 complete [ 1276.191440] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1278.220418] Key type lgssc unregistered [ 1278.445607] LNet: 56494:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1278.449657] LNetError: 56494:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1278.457575] LNet: Removed LNI 192.168.202.133@tcp [ 1279.010132] Key type .llcrypt unregistered [ 1279.011665] Key type ._llcrypt unregistered [ 1294.927699] Key type ._llcrypt registered [ 1294.929849] Key type .llcrypt registered [ 1294.991613] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1295.578672] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1295.678898] alg: No test for adler32 (adler32-zlib) [ 1296.630518] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 1296.804878] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 1298.439172] Key type lgssc registered [ 1299.028639] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1303.416434] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1303.432715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1304.798642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1307.757977] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1312.535600] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1316.758992] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1316.960451] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1320.867854] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1324.716750] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1329.192049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1329.245241] Lustre: 59209:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1329.251357] Lustre: 59209:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1329.256218] Lustre: 59209:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1329.263988] Lustre: 59209:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1329.273482] Lustre: 59209:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1329.276842] Lustre: 59209:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1333.881480] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1337.792871] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1344.480142] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1344.485488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1344.502536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1348.072780] Lustre: server umount lustre-OST0000 complete [ 1350.810753] Lustre: server umount lustre-MDT0000 complete [ 1358.823626] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1360.615143] Key type lgssc unregistered [ 1360.823666] LNet: 60820:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1360.832232] LNetError: 60820:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1360.855027] LNet: Removed LNI 192.168.202.133@tcp [ 1361.414725] Key type .llcrypt unregistered [ 1361.416963] Key type ._llcrypt unregistered [ 1381.485667] Key type ._llcrypt registered [ 1381.490564] Key type .llcrypt registered [ 1381.562265] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1382.306672] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1382.324173] alg: No test for adler32 (adler32-zlib) [ 1383.271676] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 1383.444832] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 1385.063261] Key type lgssc registered [ 1385.751704] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1390.747137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1390.763531] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1392.060487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1394.975278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1400.131914] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1404.512120] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1404.708874] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1408.648417] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1413.401940] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1416.939094] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1416.969980] Lustre: 63635:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1416.973026] Lustre: 63635:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1416.976191] Lustre: 63635:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1416.979299] Lustre: 63635:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1416.983227] Lustre: 63635:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1416.987030] Lustre: 63635:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1422.444181] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1458.145398] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1458.153544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1458.163441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1463.264194] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1463.586143] Lustre: server umount lustre-OST0000 complete [ 1465.552934] Lustre: server umount lustre-MDT0000 complete [ 1471.198080] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1472.619102] Key type lgssc unregistered [ 1472.777837] LNet: 65220:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1472.781537] LNetError: 65220:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1472.788744] LNet: Removed LNI 192.168.202.133@tcp [ 1473.237285] Key type .llcrypt unregistered [ 1473.238851] Key type ._llcrypt unregistered [ 1483.263753] Key type ._llcrypt registered [ 1483.265750] Key type .llcrypt registered [ 1483.315487] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 1492.294442] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1493.077053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1493.105864] alg: No test for adler32 (adler32-zlib) [ 1494.080411] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 1494.218183] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 1495.823190] Key type lgssc registered [ 1496.480485] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1517.161604] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1517.172336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1518.302928] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1518.326822] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1518.376833] Lustre: lustre-MDT0000: new disk, initializing [ 1518.425501] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1518.440815] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1520.816970] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1527.732656] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1527.810582] Lustre: 68349:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1527.848078] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1527.851346] Lustre: Skipped 1 previous similar message [ 1527.915243] Lustre: lustre-MDT0001: new disk, initializing [ 1527.973125] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1527.995835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1528.007684] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1530.359058] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1535.957352] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1538.725188] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1542.037682] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1542.180893] Lustre: lustre-OST0000: new disk, initializing [ 1542.182897] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1542.186572] Lustre: 70316:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1542.234510] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1542.991233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1542.999875] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1543.060934] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1545.497147] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1551.564449] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1554.198570] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1554.327491] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1556.363424] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1556.478411] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1558.496392] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1558.500415] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1558.504757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1558.508754] Lustre: Skipped 1 previous similar message [ 1558.515208] Lustre: Skipped 1 previous similar message [ 1563.648156] Lustre: server umount lustre-OST0000 complete [ 1568.737367] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1568.738240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1568.746853] Lustre: Skipped 1 previous similar message [ 1568.752780] Lustre: Skipped 1 previous similar message [ 1571.845182] Lustre: server umount lustre-MDT0000 complete [ 1574.016758] LustreError: 68341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192317 with bad export cookie 15095946596900615356 [ 1574.021596] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1574.024204] LustreError: 68341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1574.207218] Lustre: server umount lustre-MDT0001 complete [ 1579.497664] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 08:32:02 (1786192322) [ 1585.915037] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1608.153941] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 1613.023442] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1632.924135] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1633.063655] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1633.085166] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1633.127666] Lustre: lustre-MDT0000: new disk, initializing [ 1633.169709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1633.176411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1635.421387] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1641.889673] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1641.951725] Lustre: 76325:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1641.980828] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1641.983794] Lustre: Skipped 1 previous similar message [ 1642.023147] Lustre: lustre-MDT0001: new disk, initializing [ 1642.071748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1642.097096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1642.102062] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1644.486168] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1650.444441] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1653.418463] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1657.233173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1657.378280] Lustre: lustre-OST0000: new disk, initializing [ 1657.381452] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1657.386468] Lustre: 78293:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1657.432782] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1658.882742] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1658.888346] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1658.942795] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1660.324850] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1665.541569] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1668.045977] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1668.150414] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1670.292658] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1670.426759] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1674.213371] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1674.220097] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1674.221685] LustreError: Skipped 2 previous similar messages [ 1674.222648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1674.230236] Lustre: Skipped 1 previous similar message [ 1674.238535] Lustre: Skipped 1 previous similar message [ 1677.542530] Lustre: server umount lustre-OST0000 complete [ 1682.912279] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1682.916560] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1682.923806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1689.568555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1689.571480] Lustre: Skipped 3 previous similar messages [ 1694.175642] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1694.254297] Lustre: server umount lustre-MDT0000 complete [ 1694.688980] LustreError: 76338:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1694.697215] LustreError: 76338:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 1696.122489] LustreError: 76362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192439 with bad export cookie 15095946596900616238 [ 1696.123071] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1696.129072] LustreError: 76362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1696.282801] Lustre: server umount lustre-MDT0001 complete [ 1700.207623] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 08:34:03 (1786192443) [ 1703.794225] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1703.991487] LustreError: 80702:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1704.040316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1706.071877] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1709.545118] LustreError: 80702:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1709.552623] LustreError: 80702:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1711.183824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1714.111764] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1719.048936] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1722.288471] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1726.349500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1729.374687] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1733.097298] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1735.316346] Lustre: 83368:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 4fa17447-bb69-440a-b12b-d79a9027f62d at adminstrative request [ 1736.804460] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1736.814745] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1746.402950] Lustre: MGS: Client 4fa17447-bb69-440a-b12b-d79a9027f62d (at 0@lo) reconnecting [ 1746.408725] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xd17f9372b8873ba0 to 0xd17f9372b8873e40 [ 1746.420885] Lustre: MGC192.168.202.133@tcp: Connection restored to 0@lo (at 0@lo) [ 1768.891765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1768.999448] Lustre: lustre-OST0001: new disk, initializing [ 1769.002501] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1769.006871] Lustre: 83764:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1769.071281] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1769.074624] Lustre: Skipped 2 previous similar messages [ 1772.110041] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1776.182158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1776.186908] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1776.249583] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1777.805286] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1781.216825] LustreError: lustre-OST0001-osc-MDT0001: operation ost_create to node 0@lo failed: rc = -107 [ 1781.217718] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1781.230304] LustreError: Skipped 1 previous similar message [ 1781.243065] Lustre: Skipped 1 previous similar message [ 1781.248776] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1785.645456] Lustre: server umount lustre-OST0001 complete [ 1786.853965] LustreError: 82519:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1786.872510] LustreError: 82519:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1788.394022] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1788.400947] LustreError: Skipped 1 previous similar message [ 1790.437321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1790.444141] Lustre: Skipped 2 previous similar messages [ 1791.969904] LustreError: 82521:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1791.979042] LustreError: 82521:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1794.298723] Lustre: server umount lustre-OST0000 complete [ 1802.208462] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1802.211777] Lustre: Skipped 8 previous similar messages [ 1802.628450] Lustre: server umount lustre-MDT0000 complete [ 1805.054923] LustreError: 80684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192548 with bad export cookie 15095946596900617792 [ 1805.059244] LustreError: 84378:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0001-osc-MDT0001: can't precreate: rc = -5 [ 1805.060729] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1805.065202] LustreError: 80684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1805.085130] LustreError: 84378:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0001-osc-MDT0001: cannot precreate objects: rc = -5 [ 1805.268597] Lustre: server umount lustre-MDT0001 complete [ 1810.546286] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:35:53 (1786192553) [ 1811.456824] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1812.484782] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:35:55 (1786192555) [ 1813.407390] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1814.451354] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:35:57 (1786192557) [ 1815.400179] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1816.479331] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:35:59 (1786192559) [ 1817.435416] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1818.434922] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:36:01 (1786192561) [ 1819.453980] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1822.642814] loop: module loaded [ 1822.654889] loop0: detected capacity change from 0 to 4294967296 [ 1822.683307] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1823.510173] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1825.203967] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1827.383534] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1841.062304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1841.094483] LustreError: 87880:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1841.096572] LustreError: 87880:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1841.099218] LustreError: 87880:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1841.101835] LustreError: 87880:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1851.675761] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 1857.412664] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1878.408354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1878.577653] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1878.596158] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1878.663279] Lustre: lustre-MDT0000: new disk, initializing [ 1878.717590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1878.734655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1880.831256] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1886.662033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1886.715232] Lustre: 90741:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1886.857707] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1889.263404] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1895.085144] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1898.440228] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1901.703203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1901.833193] Lustre: 92706:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1901.837992] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 1905.050329] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1907.194419] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1907.225964] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1910.614927] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1912.963400] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1913.084522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1915.450487] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1915.562018] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1917.407962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1917.414201] LustreError: Skipped 1 previous similar message [ 1917.415676] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1917.424365] Lustre: Skipped 3 previous similar messages [ 1922.614163] Lustre: server umount lustre-OST0000 complete [ 1930.808856] Lustre: server umount lustre-MDT0000 complete [ 1932.629027] LustreError: 91533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192676 with bad export cookie 15095946596900618408 [ 1932.629822] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1932.635949] LustreError: 91533:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1932.845188] Lustre: server umount lustre-MDT0001 complete [ 1937.061616] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 08:38:00 (1786192680) [ 1940.788777] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1941.036814] LustreError: 95115:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1943.153455] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1947.091142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1949.353250] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1953.062619] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1955.885810] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1959.504760] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1962.440342] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1965.906016] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1967.845278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1967.964824] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1969.846595] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1974.020577] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 4 sec [ 1976.584700] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1976.654614] Lustre: lustre-OST0001: new disk, initializing [ 1976.657448] Lustre: Skipped 2 previous similar messages [ 1976.659683] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1976.662892] Lustre: Skipped 3 previous similar messages [ 1976.665764] Lustre: 98508:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1978.974341] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1981.961149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1981.965154] Lustre: Skipped 2 previous similar messages [ 1981.967302] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1982.010473] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1983.484453] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1985.209683] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1985.301978] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1987.042790] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1987.132229] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1999.907354] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2013.973399] Lustre: Failing over lustre-MDT0000 [ 2014.178697] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2014.183738] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2014.190529] Lustre: Skipped 2 previous similar messages [ 2014.194740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2014.196909] Lustre: Skipped 6 previous similar messages [ 2014.311245] Lustre: server umount lustre-MDT0000 complete [ 2015.326868] LustreError: 95110:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.33@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2015.342385] LustreError: 95110:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2031.259285] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2031.260742] LDISKFS-fs (dm-0): recovery complete [ 2031.267259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2031.332523] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2031.475104] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2031.481403] Lustre: Skipped 6 previous similar messages [ 2031.515905] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2031.716856] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2031.717268] Lustre: 96570:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2033.578693] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2036.709467] Lustre: 95116:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2036.713209] Lustre: 95116:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2036.718906] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2036.757496] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2037.071565] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2037.393778] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2037.396742] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2037.711194] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2037.715134] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2037.719387] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2037.723611] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2038.666810] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2038.671712] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2038.991199] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2038.994546] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2039.937690] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2039.940958] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2040.880097] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2040.888645] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2041.207151] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2041.210712] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2044.032313] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2044.038926] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2044.997464] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2045.001585] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2045.319431] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2045.323338] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2052.240223] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2052.244770] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2053.190233] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2053.199344] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2053.511154] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2053.514359] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2068.270116] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2068.275927] Lustre: 101354:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2069.218156] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2069.224196] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2069.543163] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2069.545978] LustreError: 101354:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2091.199366] Lustre: 101354:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2091.206512] Lustre: 101354:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2091.212439] Lustre: 101354:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b2430da7-d959-475b-9146-682479f9224d@192.168.202.33@tcp [ 2091.225144] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2091.229129] Lustre: 101354:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@0000000006d97d27 x1872958302782464/t0(8589934770) o35->b2430da7-d959-475b-9146-682479f9224d@192.168.202.33@tcp:255/0 lens 392/0 e 0 to 0 dl 1786192845 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2091.241727] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2091.248274] Lustre: lustre-MDT0000: Denying connection for new client b2430da7-d959-475b-9146-682479f9224d (at 192.168.202.33@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2091.257459] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2091.295770] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2091.297959] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2091.299304] Lustre: Skipped 2 previous similar messages [ 2091.325365] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2091.325467] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2101.217276] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2101.221555] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2101.230336] Lustre: Skipped 3 previous similar messages [ 2115.041053] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2115.099994] Lustre: server umount lustre-OST0000 complete [ 2118.628705] LustreError: 96935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2118.639501] LustreError: 96935:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 2123.182891] Lustre: server umount lustre-OST0001 complete [ 2131.582513] Lustre: server umount lustre-MDT0000 complete [ 2133.641247] LustreError: 95097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192877 with bad export cookie 15095946596900669606 [ 2133.648281] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2133.655930] LustreError: 95097:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2133.869958] Lustre: server umount lustre-MDT0001 complete [ 2137.809227] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 08:41:21 (1786192881) [ 2141.628723] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2141.637419] Lustre: *** cfs_fail_loc=197, val=0*** [ 2157.151563] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2162.271445] LustreError: 103326:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.133@tcp: failed processing log, type 4: rc = -110 [ 2167.403832] Lustre: *** cfs_fail_loc=197, val=0*** [ 2167.409630] Lustre: Skipped 65 previous similar messages [ 2190.633818] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2193.774898] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2194.615960] Lustre: server umount lustre-OST0000 complete [ 2198.004330] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 08:42:21 (1786192941) [ 2213.242061] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2213.482428] LustreError: 104943:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2215.298214] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2220.195386] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2222.620991] LustreError: 104975:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2222.626442] LustreError: 104975:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2222.630636] LustreError: 104975:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -5 [ 2222.765699] Lustre: server umount lustre-MDT0000 complete [ 2238.372724] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2242.627515] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2259.361627] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2259.459075] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2259.471586] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2259.512421] Lustre: lustre-MDT0000: new disk, initializing [ 2259.546612] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2261.319632] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2266.611844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2266.661078] Lustre: 109138:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2266.793034] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2268.529374] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2273.338206] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2275.772640] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2278.617443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2278.734756] Lustre: 111109:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2281.163819] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2285.637027] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2287.161018] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2287.458633] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2287.556868] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2289.415871] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2289.507314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2292.195378] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2292.195871] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2292.198775] LustreError: Skipped 4 previous similar messages [ 2292.210557] Lustre: Skipped 5 previous similar messages [ 2292.211374] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2292.211379] Lustre: Skipped 14 previous similar messages [ 2296.562917] Lustre: server umount lustre-OST0000 complete [ 2304.423339] Lustre: server umount lustre-MDT0000 complete [ 2306.065069] LustreError: 109133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193049 with bad export cookie 15095946596900680204 [ 2306.067277] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2306.069340] LustreError: 109133:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2306.211781] Lustre: server umount lustre-MDT0001 complete [ 2309.979556] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 08:44:13 (1786193053) [ 2313.559646] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 2314.823532] Key type lgssc unregistered [ 2314.959621] LNet: 113595:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2314.963205] LNetError: 113595:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2314.970345] LNet: Removed LNI 192.168.202.133@tcp [ 2315.376847] Key type .llcrypt unregistered [ 2315.378841] Key type ._llcrypt unregistered [ 2323.226754] Key type ._llcrypt registered [ 2323.228165] Key type .llcrypt registered [ 2323.281833] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2330.640672] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2331.066808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2331.082318] alg: No test for adler32 (adler32-zlib) [ 2331.996850] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 2332.113439] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2333.711666] Key type lgssc registered [ 2334.271687] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2352.472784] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2352.479186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2353.583303] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2353.604773] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2353.647166] Lustre: lustre-MDT0000: new disk, initializing [ 2353.668220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2353.674954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2355.281912] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2360.763544] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2360.933989] Lustre: lustre-OST0000: new disk, initializing [ 2360.936420] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2360.939394] Lustre: Skipped 1 previous similar message [ 2360.942708] Lustre: 117742:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2360.981225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2364.002392] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2367.519108] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2367.523760] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2367.539567] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2369.507253] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2369.595421] Lustre: lustre-OST0001: new disk, initializing [ 2369.598537] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2369.602320] Lustre: 118604:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2369.637413] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2372.275594] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2372.776800] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2372.784875] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2372.801829] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2378.209717] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2378.214475] Lustre: Skipped 1 previous similar message [ 2378.215542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2378.218815] Lustre: Skipped 1 previous similar message [ 2383.329795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2383.334230] Lustre: Skipped 1 previous similar message [ 2383.756068] Lustre: server umount lustre-MDT0000 complete [ 2386.307953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2388.591369] LustreError: 116926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193132 with bad export cookie 15623094336033883440 [ 2388.597045] LustreError: 116926:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2388.597110] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2388.638933] Lustre: server umount lustre-OST0000 complete [ 2390.196547] Lustre: server umount lustre-OST0001 complete [ 2393.666761] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 08:45:37 (1786193137) [ 2403.347882] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 08:45:46 (1786193146) [ 2407.972456] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2417.713440] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 08:46:01 (1786193161) [ 2434.274192] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2438.231808] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2455.064503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2455.181712] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2455.200128] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2455.241192] Lustre: lustre-MDT0000: new disk, initializing [ 2455.271927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2455.279854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2457.217053] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2462.341744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2462.394890] Lustre: 126342:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2462.416736] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2462.418932] Lustre: Skipped 1 previous similar message [ 2462.454801] Lustre: lustre-MDT0001: new disk, initializing [ 2462.497066] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2462.517920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2462.521898] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2464.163213] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2468.690760] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2471.125028] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2474.013925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2474.131485] Lustre: lustre-OST0000: new disk, initializing [ 2474.133822] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2474.137499] Lustre: 128309:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2474.175735] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2475.327257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2475.331155] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2475.375473] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2476.424304] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2480.664263] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2482.221904] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2482.316374] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2483.800035] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2483.875201] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2485.728838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2485.730815] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2485.731420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2485.731995] LustreError: Skipped 1 previous similar message [ 2485.735564] Lustre: Skipped 1 previous similar message [ 2490.717846] Lustre: server umount lustre-OST0000 complete [ 2493.409453] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2493.412718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2493.421428] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2493.424516] Lustre: Skipped 1 previous similar message [ 2495.971360] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2498.390278] Lustre: server umount lustre-MDT0000 complete [ 2499.855450] LustreError: 127137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193243 with bad export cookie 15623094336033884665 [ 2499.860393] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2499.861212] LustreError: 127137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2499.992332] Lustre: server umount lustre-MDT0001 complete [ 2503.174329] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 08:47:26 (1786193246) [ 2505.629163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2505.872649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2507.333740] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2510.747230] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2512.640324] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2515.521717] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2517.630871] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2520.105025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2522.325452] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2525.077098] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2537.654971] Lustre: *** cfs_fail_loc=159, val=0*** [ 2553.949318] Lustre: lustre-MDT0000: Client 8bdfba74-b5f5-482b-84fc-addfa9edbffe (at 192.168.202.33@tcp) reconnecting [ 2554.534613] Lustre: *** cfs_fail_loc=159, val=0*** [ 2554.536491] Lustre: Skipped 5 previous similar messages [ 2570.833221] Lustre: lustre-MDT0000: Client 8bdfba74-b5f5-482b-84fc-addfa9edbffe (at 192.168.202.33@tcp) reconnecting [ 2572.768318] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2572.768422] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2572.770279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2572.770284] Lustre: Skipped 2 previous similar messages [ 2572.770916] LustreError: Skipped 1 previous similar message [ 2572.774237] Lustre: Skipped 1 previous similar message [ 2578.259278] Lustre: server umount lustre-OST0000 complete [ 2582.495947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2582.498579] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2582.503874] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2582.506349] Lustre: Skipped 3 previous similar messages [ 2585.827218] Lustre: server umount lustre-MDT0000 complete [ 2587.226919] LustreError: 132555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193331 with bad export cookie 15623094336033885547 [ 2587.227229] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2587.230749] LustreError: 132555:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2587.368172] Lustre: server umount lustre-MDT0001 complete [ 2591.169569] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 2592.281641] Key type lgssc unregistered [ 2592.435611] LNet: 134507:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2592.439503] LNetError: 134507:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2592.446911] LNet: Removed LNI 192.168.202.133@tcp [ 2592.837265] Key type .llcrypt unregistered [ 2592.839058] Key type ._llcrypt unregistered [ 2602.488795] Key type ._llcrypt registered [ 2602.489852] Key type .llcrypt registered [ 2602.528889] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2602.901433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2602.954805] alg: No test for adler32 (adler32-zlib) [ 2603.821422] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 2603.918753] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2605.503284] Key type lgssc registered [ 2605.931319] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2609.020581] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2609.028581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2610.179633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2611.674557] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2614.970368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2615.128920] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2616.617966] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2619.237735] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2621.338486] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2623.823742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2623.944634] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2626.044729] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2628.686847] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2635.240735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2644.150835] Lustre: *** cfs_fail_loc=159, val=0*** [ 2660.435521] Lustre: lustre-MDT0000: Client d8a44dca-64e3-4449-be00-8d02b4f3526d (at 192.168.202.33@tcp) reconnecting [ 2661.267201] Lustre: *** cfs_fail_loc=159, val=0*** [ 2677.847991] Lustre: lustre-MDT0001: Client d8a44dca-64e3-4449-be00-8d02b4f3526d (at 192.168.202.33@tcp) reconnecting [ 2678.479617] Lustre: *** cfs_fail_loc=159, val=0*** [ 2678.482231] Lustre: Skipped 3 previous similar messages [ 2694.741428] Lustre: lustre-MDT0001: Client d8a44dca-64e3-4449-be00-8d02b4f3526d (at 192.168.202.33@tcp) reconnecting [ 2696.050813] Lustre: 138953:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2696.965262] Lustre: *** cfs_fail_loc=159, val=0*** [ 2696.968671] Lustre: Skipped 2 previous similar messages [ 2713.158930] Lustre: lustre-MDT0000: Client 49f11b5b-b248-4a9b-9c4f-2e9fab5962e2 (at 192.168.202.33@tcp) reconnecting [ 2713.839327] Lustre: *** cfs_fail_loc=159, val=0*** [ 2713.845969] Lustre: Skipped 14 previous similar messages [ 2729.043978] Lustre: lustre-MDT0000: Client 49f11b5b-b248-4a9b-9c4f-2e9fab5962e2 (at 192.168.202.33@tcp) reconnecting [ 2729.652259] Lustre: 139243:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2732.514621] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2732.514812] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2732.515695] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2732.519201] Lustre: Skipped 1 previous similar message [ 2732.522571] LustreError: Skipped 1 previous similar message [ 2737.247787] Lustre: server umount lustre-OST0000 complete [ 2742.752691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2742.753342] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2742.756391] Lustre: Skipped 1 previous similar message [ 2742.758982] Lustre: Skipped 1 previous similar message [ 2744.928925] Lustre: server umount lustre-MDT0000 complete [ 2746.396179] LustreError: 138916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193490 with bad export cookie 10729693427087949286 [ 2746.398141] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2746.400286] LustreError: 138916:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2746.513135] Lustre: server umount lustre-MDT0001 complete [ 2750.822576] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 2752.026970] Key type lgssc unregistered [ 2752.170481] LNet: 140295:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2752.173921] LNetError: 140295:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2752.189323] LNet: Removed LNI 192.168.202.133@tcp [ 2752.582865] Key type .llcrypt unregistered [ 2752.584340] Key type ._llcrypt unregistered [ 2762.103256] Key type ._llcrypt registered [ 2762.104390] Key type .llcrypt registered [ 2762.138032] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2762.535510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2762.557932] alg: No test for adler32 (adler32-zlib) [ 2763.410654] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 2763.499536] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2765.079192] Key type lgssc registered [ 2765.430327] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2768.137675] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2768.145165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2769.286779] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2770.631357] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2773.665991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2773.795295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2775.217458] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2778.057510] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2780.181423] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2782.758843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2782.877216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2783.911583] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2785.099259] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2787.879111] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2794.151803] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2801.016548] Lustre: 144289:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2801.675251] Lustre: 144337:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2802.200751] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2803.463225] Lustre: 144483:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2803.807339] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2803.812120] Lustre: Skipped 1 previous similar message [ 2804.136074] Lustre: 144582:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2807.254432] Lustre: server umount lustre-OST0000 complete [ 2809.824889] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2809.826647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2809.826929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2809.829579] Lustre: Skipped 1 previous similar message [ 2814.944283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2814.947307] Lustre: Skipped 2 previous similar messages [ 2819.064912] Lustre: server umount lustre-MDT0000 complete [ 2820.065344] LustreError: 144179:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2820.072511] LustreError: 144179:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2820.321205] LustreError: 142419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193564 with bad export cookie 10666303955447899846 [ 2820.324302] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2820.325604] LustreError: 142419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2830.873281] Lustre: server umount lustre-MDT0001 complete [ 2834.790478] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 2835.861468] Key type lgssc unregistered [ 2835.999347] LNet: 145639:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2836.001930] LNetError: 145639:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2836.008304] LNet: Removed LNI 192.168.202.133@tcp [ 2836.306954] Key type .llcrypt unregistered [ 2836.308519] Key type ._llcrypt unregistered [ 2846.273186] Key type ._llcrypt registered [ 2846.274357] Key type .llcrypt registered [ 2846.312588] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2846.629192] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2846.654925] alg: No test for adler32 (adler32-zlib) [ 2847.517746] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 2847.617930] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2849.215169] Key type lgssc registered [ 2849.557511] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2852.197129] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2852.204041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2853.360127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2854.781346] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2857.918176] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2858.066346] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2859.615650] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2862.240797] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2864.175639] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2866.448473] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2866.570970] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2867.623374] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2868.503111] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2870.997176] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2878.884616] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2883.701402] Lustre: *** cfs_fail_loc=159, val=0*** [ 2899.026981] Lustre: lustre-MDT0000: Client 63805243-f10d-45f0-ab48-204106910a64 (at 192.168.202.33@tcp) reconnecting [ 2918.879988] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2918.882863] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2918.887697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2919.905710] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2919.906443] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2919.906816] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2921.045370] Lustre: server umount lustre-OST0000 complete [ 2924.512141] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2924.514760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2924.518655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2924.520870] Lustre: Skipped 1 previous similar message [ 2928.405481] Lustre: server umount lustre-MDT0000 complete [ 2929.641278] LustreError: 147016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193673 with bad export cookie 11417123403060505093 [ 2929.641842] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2929.646110] LustreError: 147016:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2929.758851] Lustre: server umount lustre-MDT0001 complete [ 2933.149602] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 2934.155579] Key type lgssc unregistered [ 2934.275346] LNet: 150726:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2934.277992] LNetError: 150726:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2934.290297] LNet: Removed LNI 192.168.202.133@tcp [ 2934.568132] Key type .llcrypt unregistered [ 2934.569301] Key type ._llcrypt unregistered [ 2943.070961] Key type ._llcrypt registered [ 2943.072684] Key type .llcrypt registered [ 2943.114764] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2943.461167] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2943.491083] alg: No test for adler32 (adler32-zlib) [ 2944.356645] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 2944.447964] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2946.023147] Key type lgssc registered [ 2946.373464] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2948.763216] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2948.768605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2949.917572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2951.249905] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2954.023821] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2954.154843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2955.503319] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2957.801945] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2959.510551] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2961.686837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2961.819177] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2963.653316] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2963.877407] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2965.981124] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2975.141863] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2989.004059] Lustre: lustre-OST0000: evicting 37171980-89bf-498f-b15e-e18439987113 (at 192.168.202.33@tcp) by administrative request [ 2989.007713] Lustre: Skipped 1 previous similar message [ 3015.136391] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3015.140126] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3015.145990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3016.161722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3016.164505] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3016.165205] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3016.173106] Lustre: Skipped 1 previous similar message [ 3019.348228] Lustre: server umount lustre-OST0000 complete [ 3020.767792] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3020.771327] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3020.777800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3026.400748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3026.402793] Lustre: Skipped 3 previous similar messages [ 3031.520070] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3031.522048] Lustre: Skipped 1 previous similar message [ 3035.103151] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3035.154822] Lustre: server umount lustre-MDT0000 complete [ 3036.358082] LustreError: 152099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193780 with bad export cookie 13115705414984604431 [ 3036.360943] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3036.361215] LustreError: 152099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3036.472797] Lustre: server umount lustre-MDT0001 complete [ 3039.966506] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3040.959917] Key type lgssc unregistered [ 3041.085336] LNet: 156207:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3041.088091] LNetError: 156207:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3041.094305] LNet: Removed LNI 192.168.202.133@tcp [ 3041.385505] Key type .llcrypt unregistered [ 3041.387171] Key type ._llcrypt unregistered [ 3065.479713] Key type ._llcrypt registered [ 3065.480776] Key type .llcrypt registered [ 3065.517199] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3065.932140] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3065.954728] alg: No test for adler32 (adler32-zlib) [ 3066.808230] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 3066.895727] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3068.479201] Key type lgssc registered [ 3068.838127] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3071.621536] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3071.630622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3072.813673] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3074.192438] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3077.076069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3077.208531] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3078.590920] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3081.022069] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3082.797963] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3085.015492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3085.134083] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3087.081675] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3087.205413] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3089.535829] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3098.470083] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3103.713860] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3103.714305] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3103.715143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3103.718032] Lustre: Skipped 1 previous similar message [ 3104.844320] Lustre: server umount lustre-OST0000 complete [ 3107.808060] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3107.810219] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3107.814825] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3107.816911] Lustre: Skipped 1 previous similar message [ 3108.248640] Lustre: server umount lustre-MDT0000 complete [ 3108.832064] LustreError: 158668:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3108.832546] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3108.837788] LustreError: 158668:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3109.463160] LustreError: 158648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193853 with bad export cookie 2815646079153822122 [ 3109.466360] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3109.468155] LustreError: 158648:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3109.597808] Lustre: server umount lustre-MDT0001 complete [ 3113.316770] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3114.266407] Key type lgssc unregistered [ 3114.397299] LNet: 162227:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3114.399871] LNetError: 162227:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3114.406452] LNet: Removed LNI 192.168.202.133@tcp [ 3114.713424] Key type .llcrypt unregistered [ 3114.714587] Key type ._llcrypt unregistered [ 3124.401837] Key type ._llcrypt registered [ 3124.403491] Key type .llcrypt registered [ 3124.450760] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3124.882616] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3124.922499] alg: No test for adler32 (adler32-zlib) [ 3125.769543] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 3125.853752] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3127.431171] Key type lgssc registered [ 3127.804568] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3132.667170] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 08:57:56 (1786193876) [ 3137.736205] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 3140.850658] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3152.370587] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3152.375609] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3153.449959] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3153.459924] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3153.499296] Lustre: lustre-MDT0000: new disk, initializing [ 3153.517704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3153.522947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3154.777964] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3158.723730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3158.762842] Lustre: 166460:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3158.785443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3158.787084] Lustre: Skipped 1 previous similar message [ 3158.826257] Lustre: lustre-MDT0001: new disk, initializing [ 3158.860425] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3158.873091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3158.876164] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3160.237124] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3163.876711] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3165.676170] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3167.808326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3167.899972] Lustre: lustre-OST0000: new disk, initializing [ 3167.901609] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3167.904063] Lustre: 168425:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3167.929214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3169.803793] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3173.429678] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3174.900734] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3174.903520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3174.936104] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3185.122232] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3185.122540] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3185.124395] LustreError: Skipped 1 previous similar message [ 3185.128640] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3185.131097] Lustre: Skipped 1 previous similar message [ 3188.110560] Lustre: server umount lustre-OST0000 complete [ 3189.727732] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3189.729742] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3189.732659] Lustre: Skipped 1 previous similar message [ 3189.734437] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3195.360205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3195.362053] Lustre: Skipped 3 previous similar messages [ 3195.413871] Lustre: server umount lustre-MDT0000 complete [ 3196.632112] LustreError: 166452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193940 with bad export cookie 16079376448566114979 [ 3196.634859] LustreError: 166452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3196.635328] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3196.745461] Lustre: server umount lustre-MDT0001 complete [ 3200.090995] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3201.024672] Key type lgssc unregistered [ 3201.143805] LNet: 170331:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3201.146111] LNetError: 170331:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3201.152252] LNet: Removed LNI 192.168.202.133@tcp [ 3201.436791] Key type .llcrypt unregistered [ 3201.437724] Key type ._llcrypt unregistered [ 3209.953500] Key type ._llcrypt registered [ 3209.954360] Key type .llcrypt registered [ 3209.990505] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3210.303656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3210.354717] alg: No test for adler32 (adler32-zlib) [ 3211.205431] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 3211.287867] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3212.863155] Key type lgssc registered [ 3213.209714] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3215.571670] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3215.577302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3216.732824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3217.930117] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3220.584274] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3220.723090] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3222.133514] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3224.466923] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3226.253646] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3228.491048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3228.597739] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3230.389483] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3232.684855] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3241.896940] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3245.377162] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3245.384332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3245.387722] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3245.389620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.394163] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3245.718943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.029584] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3246.031882] Lustre: Skipped 1 previous similar message [ 3246.036867] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3246.040132] Lustre: Skipped 1 previous similar message [ 3246.041351] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3246.043317] Lustre: Skipped 1 previous similar message [ 3246.044648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.048152] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3246.050126] Lustre: Skipped 1 previous similar message [ 3246.330746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.620089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.899316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.199806] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3247.202368] Lustre: Skipped 3 previous similar messages [ 3247.208297] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3247.211253] Lustre: Skipped 3 previous similar messages [ 3247.212573] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3247.214532] Lustre: Skipped 3 previous similar messages [ 3247.215757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.219371] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3247.221220] Lustre: Skipped 3 previous similar messages [ 3247.509860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.793211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.081742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.395995] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3248.398570] LustreError: 171741:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3248.400308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.680958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.983594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.272504] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3249.274359] Lustre: Skipped 6 previous similar messages [ 3249.279599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3249.282632] Lustre: Skipped 6 previous similar messages [ 3249.284276] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3249.285744] Lustre: Skipped 6 previous similar messages [ 3249.286934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.291994] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3249.294713] Lustre: Skipped 6 previous similar messages [ 3249.603341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.900964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.205716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.505971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.812456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.118348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.458308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.774798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.098940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.399684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.708906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.011070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.334702] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3253.337188] Lustre: Skipped 12 previous similar messages [ 3253.343495] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3253.347081] Lustre: Skipped 12 previous similar messages [ 3253.348614] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3253.350666] Lustre: Skipped 12 previous similar messages [ 3253.352145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.356908] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3253.359170] Lustre: Skipped 12 previous similar messages [ 3253.664101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.991058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.293606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.616824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.929639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.241550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.535727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.859918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.163000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.460958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.764034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.070303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.353177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.650578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.924160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.203979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.489235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3258.774431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.063191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.368894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.655082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.950248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.241235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.551072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.844678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.148703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.451527] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3261.454282] Lustre: Skipped 26 previous similar messages [ 3261.461334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3261.466336] Lustre: Skipped 26 previous similar messages [ 3261.468448] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3261.470690] Lustre: Skipped 26 previous similar messages [ 3261.471941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.475146] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3261.477493] Lustre: Skipped 26 previous similar messages [ 3261.791846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.090602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.382339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.687776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.017644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.305190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.611981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.913965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.235125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.533851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.830672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.109027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.409458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.687069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.962458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.235290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.519544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.792294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.066588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.352925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.640834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.912761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.185891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.473077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.775625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.061303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.355073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.627375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.902639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.176202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.473642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3270.763209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.064850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.353366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.662284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.967432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.303615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.594439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.913819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.244637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.592342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.971323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.340314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.692981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.044819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.417898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.748482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.054677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.363900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3276.695050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.094950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.452432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.800614] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3277.803913] Lustre: Skipped 52 previous similar messages [ 3277.810329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3277.815285] Lustre: Skipped 52 previous similar messages [ 3277.817707] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3277.820374] Lustre: Skipped 52 previous similar messages [ 3277.821939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.826455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3277.828960] Lustre: Skipped 52 previous similar messages [ 3278.167362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.502812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.858413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.215227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.533593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.848573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.175939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.495397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.807709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.118430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.439549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.774271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.083750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.420392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.735945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.052331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.356766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.636917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.919693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.219679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.513469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.796843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.072928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.377658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.671097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3285.973851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.257253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.560421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.859561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.153682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.470414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.775613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.073406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.377557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.679581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3288.986225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.296914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.626680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.949714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.279860] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3290.283172] LustreError: 171741:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3290.284451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.592168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.917365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.237430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.579655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.891801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.211526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.521759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.845736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.162440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.477659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.785514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.114588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.442213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.782414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.117397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.425783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.735796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.027241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.334654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.657691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.962310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.272581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.598066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.960024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.298495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.627762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.975129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.324753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.644221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.955132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.265166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.587683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.909544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.251556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.599409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.972239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.284669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.608166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.918723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.237255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.541267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.861421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.177886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.506469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.853282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.171880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.491990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3305.820760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.118737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.421665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.742909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.089663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.427307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.758911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.066431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.368413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.661136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.958055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.261426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.573521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.866799] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3309.868799] Lustre: Skipped 100 previous similar messages [ 3309.874825] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3309.880576] Lustre: Skipped 100 previous similar messages [ 3309.882634] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3309.884751] Lustre: Skipped 100 previous similar messages [ 3309.886886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.892621] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3309.895619] Lustre: Skipped 100 previous similar messages [ 3310.184506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.476311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.800197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.092632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.386728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.688615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.004594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.298983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.588946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.875487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.175029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.456965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.754503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.050786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.356228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.650138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.938503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.225313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.506132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.789125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.069606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.355773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.664126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.961977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.269988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.574916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.888997] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.176328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.471454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.777469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.098605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.400775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.715581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.992282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.291323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.577971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.858657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.137070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.427653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.705678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.981496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.259168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.558102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.849073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.135669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.441875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.753934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.059736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.364982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.666841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.966406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.247608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.527385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.796766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.085767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.365398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.649804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.930706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.215992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.501118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.778257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.078507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.398720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.690402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.977268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.264849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.571381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.873715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.187152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.490262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.799961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.086705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.372749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.651313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.937911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.222846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.527925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.837068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.157642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.481866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.793969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.121825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.464180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.774423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.083401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.391046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3335.709174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.008140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.305746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.598762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.897964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.190154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.488326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.778870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.079700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.368411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.673395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.974671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.295097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.623042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.938210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.260708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.586910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.887494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.179819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.476085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.773909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.063457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.355885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.638763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.925645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.208441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.498409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.792772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.098272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.385242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.674837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.973191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.295123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.607212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.916272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.213625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.533697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.823040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.113519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.410357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.702619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.997614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.289788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.584192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.900725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.209878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.519131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.816287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.138769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.447163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.737485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.041826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.354308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.640155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.923538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.221277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.549283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.844720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.152664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.460869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.786680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.095632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.421257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.756634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.102085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.417123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.748090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.064628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.385520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.697824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.019958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.338451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.649545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.944657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.246810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.566988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.900129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.215799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.524575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.838912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.144337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.438494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.733681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.042547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.349772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.659048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.963266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.266322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.578268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.884777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.177822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.464829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.771290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.102461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.425446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.735561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.086539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.421277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.719859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.030177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.376296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.688282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.016301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.332561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.660945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.961980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.279341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.595614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.936016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.277460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.590931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.895142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.202418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.501354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.799322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.108657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.432979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3371.721702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.039452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.339962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.666074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.966851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.267975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.563776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.888718] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3373.891705] Lustre: Skipped 210 previous similar messages [ 3373.897601] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3373.900693] Lustre: Skipped 210 previous similar messages [ 3373.901986] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3373.903916] Lustre: Skipped 210 previous similar messages [ 3373.905559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.908542] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3373.910205] Lustre: Skipped 210 previous similar messages [ 3374.225218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.552432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.853309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.153215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.439360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.727253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.022077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.335235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.622758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.921711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.219728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.545427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.832238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.121519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.404543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.698179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.994666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.312439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.632710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.966249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.252595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.543981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.826955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.115556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.393940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.682357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.970562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.288213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.580223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.869822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.169280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.481977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.774083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.069670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.368733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.687485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.981574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.287715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.588963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.886784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.174125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.480155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.778199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.078606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.369940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.656174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.942255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.250834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.538185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.819313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.103399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.426669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.743814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.045916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.351438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.669016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.979781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.281612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.578078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.908439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.224113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.538491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.850163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.181471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.501478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.791742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.092123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.392272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.687230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.982666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.263496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.581350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.876732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.179222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.482870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.780523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.060842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.345068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.638357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.940728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.241854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.545087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.844617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.170472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.469858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.786891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.112681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.442356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.733042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.028176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.334230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.645759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.933299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.225383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.521184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.837450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.120495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.415805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.712323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.028275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.337620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.631311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.935158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.235687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.509671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.786609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.061460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.349784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.649760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.930622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.217979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.533406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.819078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.103500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.394037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.710643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.013143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.317661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.611593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.916566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.222332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.549090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.852448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.168151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.481824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.786696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.084683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.379485] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3412.382351] LustreError: 171741:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3412.383921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.677663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.979387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.274852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.599843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.895360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.178428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.457964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.747215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.059683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.365930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.669215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.970228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.268319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.573572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.866792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.167834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.469790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.762770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.040866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.348534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.626557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.912592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.215524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.538751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.829458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.126443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.427588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.732721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.022263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.308307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.600350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.902414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.196429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.491531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.776233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.076256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.363885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.664237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.973377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.284239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.567702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.859107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.144336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.475802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.789714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.082784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.355955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.648850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.939539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.221438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.492561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.787180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.067785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.360986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.631964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.906882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.180846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.453677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.724166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.021534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.304743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.583456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.865740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.165593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.456815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.757451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.048402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.344992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.625976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.907219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.179423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.477402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.793321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.093277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.383139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.686708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.976631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.271465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.559068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.858437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.148269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.431407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.714488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.010858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.294115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.575418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.856674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.157135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.443858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.718066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.996417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.284032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.577799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.872540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.156988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.439948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.719559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.005512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.285283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.571620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.845604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.120984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.397996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.696586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.995135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.300941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.607604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.893138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.173670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.448653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.729414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.006878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.281839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.588950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.871206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.162190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.443228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.722267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.997260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.290340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.578890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.875434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.162766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.468876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.782356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.085678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.382704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.697617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.993936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.291135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.605391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.946679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.264167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.581394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.895860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.200776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.515529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.821870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.135871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.458951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.771443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.075687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.378475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.717378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.027483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.351608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.653318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.973883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.279880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.581072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.878781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.216990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.557981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.882109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.179234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.518883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.825187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.130334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.436152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.756588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.061868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.363440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.692758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.038207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.357679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.664897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.987158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.334874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.651191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.984756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.306430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.660727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.986141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.309563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.628294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3464.958100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.277467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.589975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.899847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.259127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.594989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.927927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.258715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.621787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.953539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.292952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.611541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.969456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.305643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.655669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.005859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.370961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.699564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.041800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.372558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.713047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.021902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.312069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.599124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.900948] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3472.904850] LustreError: 171741:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3472.906094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.215621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.500456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.780617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.074805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.345324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.641698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.947185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.264789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.576200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.884757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.197712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.512569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.816341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.149978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.464345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.780617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.104628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.433437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.740404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.078263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.405833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.735725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.073849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.430301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.759592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.086948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.414082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.756348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.077485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.402919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.706839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.031438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.343088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.665592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.985665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.321986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.634132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.958565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.276809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.602716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3485.916406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.222950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.530295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.866778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.196995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.518879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.841507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.187956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.519173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.859787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.178706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.523935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.847957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.185269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.520385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.850651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.146708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.455791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.761752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.080569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.398350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.709079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.009192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.328298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.638241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.941313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.233654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.544536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.833784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.123905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.405132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.693175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.968910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.263923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.538566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.825748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.091864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.355095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.634039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.933363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.235484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.535303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.833265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.153071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.448528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.756700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.062025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.387100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3500.695089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.002496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.311306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.639470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.944302] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3501.946535] Lustre: Skipped 420 previous similar messages [ 3501.952123] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3501.956175] Lustre: Skipped 420 previous similar messages [ 3501.958230] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3501.961358] Lustre: Skipped 420 previous similar messages [ 3501.963239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.966648] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3501.968719] Lustre: Skipped 420 previous similar messages [ 3502.279383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.585530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.914213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.227820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.543266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.866222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.196364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.509764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.825957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.144521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.483307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.819235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.160073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.473732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.793945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.098487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.413334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.734072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.067665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.373888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.695499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.004191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.325437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.634704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.945223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.239658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.561688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.860137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.158572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.462934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.779778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.081068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.377026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.675814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.002043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.313846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.623607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.922601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.230208] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3514.233409] LustreError: 171741:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3514.235714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.538826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.857154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.181283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.532526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.865178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.207635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.526451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.879847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.196366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.514591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.815458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.152378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.462400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.770029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.081485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.424467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.736955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.051307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.361557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.692839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.001027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.315286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.615839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.925870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.222288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.523211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.821526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.134647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.422872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.721818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.022631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.353167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.642251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.940918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.244684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.559334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.858676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.157460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.465514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.787931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.106114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.429505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.742106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.067950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.372704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.680418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.988176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.313573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.629921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.948380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.264860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.620608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.935434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.253462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.578224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.932796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.268130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.584523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.899344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.219013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.525427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.842018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.148675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.489290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.807785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.131187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.450700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.797295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.111309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.424760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.724608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.041271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.336688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.646429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.960773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.280702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.594508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.911419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.238339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.579809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.909544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.277462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.605365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.938401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.245519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.550600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.862073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.179861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.482150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.781085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.108915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.459597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.757855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.071950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.378371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.694962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.999611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.293370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.593799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.908991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.214647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.531636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.830622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.139659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.450311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.752527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.062155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.364666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.664995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.956928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.250766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.574190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.872860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.183338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.484497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.799420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.110083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.411136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.711280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.032650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.330937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.577977] LustreError: 173541:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3618.580236] LustreError: 171741:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3681.247926] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3681.251870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3682.272915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3682.275769] Lustre: Skipped 1 previous similar message [ 3686.975469] Lustre: server umount lustre-OST0000 complete [ 3691.999996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3692.004182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3694.550609] Lustre: server umount lustre-MDT0000 complete [ 3695.831083] LustreError: 171747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194439 with bad export cookie 5224248675280449266 [ 3695.834200] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3695.834737] LustreError: 171747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3695.963955] Lustre: server umount lustre-MDT0001 complete [ 3699.762215] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3700.835389] Key type lgssc unregistered [ 3700.969477] LNet: 226761:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3700.973265] LNetError: 226761:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3700.981364] LNet: Removed LNI 192.168.202.133@tcp [ 3701.316970] Key type .llcrypt unregistered [ 3701.318602] Key type ._llcrypt unregistered [ 3710.669586] Key type ._llcrypt registered [ 3710.670841] Key type .llcrypt registered [ 3710.709276] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3711.052353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3711.092405] alg: No test for adler32 (adler32-zlib) [ 3711.949558] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 3712.046777] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3713.623137] Key type lgssc registered [ 3714.000386] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3716.600683] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3716.607257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3717.748855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3719.143920] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3722.045567] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3722.182121] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3723.649456] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3726.181387] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3728.120727] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3730.564651] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3730.707858] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3732.829764] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3735.405931] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3743.014533] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3747.643922] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3748.199641] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3748.203035] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3748.207067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.210177] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3748.210192] LustreError: 228172:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3750.990253] Lustre: lustre-OST0000: Not available for connect from 192.168.202.33@tcp (stopping) [ 3753.441174] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3753.443462] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3753.446139] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3753.450121] Lustre: Skipped 1 previous similar message [ 3756.101783] Lustre: lustre-OST0000: Not available for connect from 192.168.202.33@tcp (stopping) [ 3756.104471] Lustre: Skipped 1 previous similar message [ 3758.560711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3758.562849] Lustre: Skipped 1 previous similar message [ 3763.167135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3763.225400] Lustre: server umount lustre-OST0000 complete [ 3768.287112] LustreError: 228172:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3791.229260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3791.339843] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3793.266729] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3795.721715] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3796.239604] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 3798.055245] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f5c431f2000.ost_server_uuid 50 [ 3798.590321] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f5c431f2000.ost_server_uuid in FULL state after 0 sec [ 3805.710973] Lustre: server umount lustre-OST0000 complete [ 3809.247911] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3809.252071] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3809.258797] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3809.261661] Lustre: Skipped 1 previous similar message [ 3813.147199] Lustre: server umount lustre-MDT0000 complete [ 3814.355211] LustreError: 228132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194558 with bad export cookie 16263164591684436426 [ 3814.358668] LustreError: 228132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3814.358775] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3814.484878] Lustre: server umount lustre-MDT0001 complete [ 3818.029780] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3819.064447] Key type lgssc unregistered [ 3819.188301] LNet: 233376:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3819.190480] LNetError: 233376:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3819.197315] LNet: Removed LNI 192.168.202.133@tcp [ 3819.488353] Key type .llcrypt unregistered [ 3819.489535] Key type ._llcrypt unregistered [ 3830.592670] Key type ._llcrypt registered [ 3830.593977] Key type .llcrypt registered [ 3830.632733] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3830.974613] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3831.002602] alg: No test for adler32 (adler32-zlib) [ 3831.847938] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 3831.931788] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3833.511136] Key type lgssc registered [ 3833.914373] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3837.046179] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3837.051812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3838.200917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3839.484435] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3842.245236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3842.372186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3843.717035] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3844.675099] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3846.886558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3847.023815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3848.978430] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3849.065854] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3852.076161] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3852.150970] Lustre: lustre-OST0001: new disk, initializing [ 3852.152751] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3852.155730] Lustre: 237389:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3852.194224] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3853.059254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3853.063370] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3853.105437] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3854.077216] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3859.505755] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3862.086476] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3869.666021] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3869.668843] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3869.670063] Lustre: Skipped 2 previous similar messages [ 3869.673065] Lustre: Skipped 1 previous similar message [ 3874.528349] Lustre: server umount lustre-MDT0000 complete [ 3874.785362] LustreError: 235075:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3874.791438] LustreError: 235075:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3875.748635] LustreError: 235060:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194619 with bad export cookie 18356434436403912361 [ 3875.753518] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3875.883736] Lustre: server umount lustre-MDT0001 complete [ 3888.132727] Lustre: server umount lustre-OST0000 complete [ 3899.681146] Lustre: server umount lustre-OST0001 complete [ 3906.707772] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3910.204428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3910.342154] LustreError: 240497:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3910.359932] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3911.770667] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3914.583925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3916.098596] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3917.080304] Lustre: 241639:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3919.301690] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3919.426627] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3919.429447] Lustre: Skipped 1 previous similar message [ 3920.483827] LustreError: 241993:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3920.488080] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 3921.389360] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3924.321608] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3926.284283] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3929.420243] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3930.583234] Lustre: 243506:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3935.199928] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3935.202246] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3935.206231] Lustre: Skipped 1 previous similar message [ 3935.208266] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3935.211274] Lustre: Skipped 2 previous similar messages [ 3939.930268] Lustre: server umount mylustre-MDT0000 complete [ 3940.834545] LustreError: 240497:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3940.842181] LustreError: 240497:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3941.133447] LustreError: 243508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194684 with bad export cookie 18356434436403915126 [ 3941.136136] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3941.138874] LustreError: 243508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3941.261322] Lustre: server umount mylustre-MDT0001 complete [ 3953.154352] Lustre: server umount mylustre-OST0000 complete [ 3964.698051] Lustre: server umount mylustre-OST0001 complete [ 3972.009897] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3975.668141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3975.823875] LustreError: 245811:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3975.843499] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3975.845482] Lustre: Skipped 1 previous similar message [ 3977.249100] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3980.176912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3981.689655] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3982.657028] Lustre: 246951:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3984.858379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3986.021442] LustreError: 247306:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3986.026906] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 3986.928744] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3989.835811] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3991.985126] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3995.262369] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3996.538247] Lustre: 248821:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4000.736163] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4000.738869] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4000.743670] Lustre: Skipped 3 previous similar messages [ 4000.745762] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 4000.747442] Lustre: Skipped 3 previous similar messages [ 4004.701358] Lustre: server umount tfs-MDT0000 complete [ 4005.862053] LustreError: 248131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194749 with bad export cookie 18356434436403916792 [ 4005.865330] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4005.866422] LustreError: 248131:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4005.981060] Lustre: server umount tfs-MDT0001 complete [ 4017.411575] Lustre: server umount tfs-OST0000 complete [ 4028.893712] Lustre: server umount tfs-OST0001 complete [ 4035.946822] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4039.495748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4039.644339] LustreError: 251067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4039.662452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4039.664776] Lustre: Skipped 3 previous similar messages [ 4041.009349] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4043.858207] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4045.329902] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4046.302291] Lustre: 252208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 4048.485053] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4049.640275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 4050.540859] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4053.444678] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4055.608510] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4058.770799] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 4059.961876] Lustre: 254076:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 4062.064850] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 09:13:25 (1786194805) [ 4074.977831] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4074.978262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.981240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.983975] Lustre: Skipped 6 previous similar messages [ 4074.985662] Lustre: Skipped 6 previous similar messages [ 4079.898729] Lustre: server umount lustre-MDT0000 complete [ 4080.097314] LustreError: 251800:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4080.102837] LustreError: 251800:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 4081.136517] LustreError: 254079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194825 with bad export cookie 18356434436403918213 [ 4081.140428] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4081.143580] LustreError: 254079:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4085.217254] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4085.218051] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4085.220760] Lustre: Skipped 1 previous similar message [ 4085.222264] Lustre: Skipped 1 previous similar message [ 4087.340988] Lustre: server umount lustre-MDT0001 complete [ 4094.652713] Lustre: server umount lustre-OST0000 complete [ 4110.303127] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4110.370493] Lustre: server umount lustre-OST0001 complete [ 4112.396292] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 4115.453442] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4126.843475] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 4126.843475] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4126.843475] [ 4126.850547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4126.929571] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4126.941941] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4126.981236] Lustre: lustre-MDT0000: new disk, initializing [ 4126.999931] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4127.002017] Lustre: Skipped 3 previous similar messages [ 4127.006396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4128.177056] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4132.217140] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 4132.217140] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 4132.217140] [ 4132.223957] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 4132.273429] Lustre: 257255:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4132.295986] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4132.297804] Lustre: Skipped 1 previous similar message [ 4132.333852] Lustre: lustre-MDT0001: new disk, initializing [ 4132.382083] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4132.386394] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4133.726344] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4137.879049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4137.989686] Lustre: lustre-OST0000: new disk, initializing [ 4137.992263] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4137.995603] Lustre: 258839:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4139.950975] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4139.963120] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4139.968606] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4140.006928] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4144.187975] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4144.255759] Lustre: lustre-OST0001: new disk, initializing [ 4144.257551] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4144.260512] Lustre: 259711:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4146.240915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 4146.244056] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 4146.278345] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 4146.316499] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4150.467151] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 09:14:54 (1786194894) [ 4153.840667] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 09:14:57 (1786194897) [ 4165.600377] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4165.600447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4165.603624] LustreError: Skipped 1 previous similar message [ 4165.610315] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4165.613506] Lustre: Skipped 4 previous similar messages [ 4171.332506] Lustre: server umount lustre-OST0000 complete [ 4171.748336] LustreError: 259202:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4171.753042] LustreError: 259202:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 4173.279822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4178.906257] Lustre: server umount lustre-MDT0000 complete [ 4180.175268] LustreError: 257248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194924 with bad export cookie 18356434436403919543 [ 4180.178994] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4180.181567] LustreError: 257248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4181.984448] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4181.988449] Lustre: Skipped 4 previous similar messages [ 4181.990561] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4181.992652] Lustre: Skipped 6 previous similar messages [ 4186.348562] Lustre: server umount lustre-MDT0001 complete [ 4190.088493] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4259.545437] Lustre: server umount lustre-OST0001 complete [ 4261.563653] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 4267.983246] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4279.749597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4279.827466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4279.837439] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4279.870941] Lustre: lustre-MDT0000: new disk, initializing [ 4279.891679] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4279.894378] Lustre: Skipped 3 previous similar messages [ 4279.898916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4281.257305] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4285.184826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4285.225659] Lustre: 264480:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4285.326321] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4286.597509] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4290.171266] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4291.978301] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4294.090285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4294.191831] Lustre: 266446:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4295.609359] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4295.644886] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4295.995988] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4299.479273] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4305.888954] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4305.890886] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4305.892342] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4305.893846] Lustre: Skipped 1 previous similar message [ 4305.896377] Lustre: Skipped 1 previous similar message [ 4307.852870] Lustre: server umount lustre-OST0000 complete [ 4323.295114] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4323.358315] Lustre: server umount lustre-MDT0000 complete [ 4324.612227] LustreError: 264474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195068 with bad export cookie 18356434436403921013 [ 4324.616052] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4324.616397] LustreError: 264474:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4324.744459] Lustre: server umount lustre-MDT0001 complete [ 4328.679252] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4329.743358] Key type lgssc unregistered [ 4329.886462] LNet: 268208:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4329.889460] LNetError: 268208:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4329.896316] LNet: Removed LNI 192.168.202.133@tcp [ 4330.217147] Key type .llcrypt unregistered [ 4330.219130] Key type ._llcrypt unregistered [ 4340.708708] Key type ._llcrypt registered [ 4340.709751] Key type .llcrypt registered [ 4340.747613] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4341.054104] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4341.066937] alg: No test for adler32 (adler32-zlib) [ 4341.920886] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 4342.013179] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4343.591103] Key type lgssc registered [ 4343.935879] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4346.445932] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4346.454043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4347.595775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4348.907593] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4351.694347] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4351.814634] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4353.109575] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4355.469963] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4357.420263] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4359.752431] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4359.876682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4361.860802] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4364.311602] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4364.638970] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 4367.329803] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4367.333788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4367.334155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4371.345234] Lustre: server umount lustre-OST0000 complete [ 4377.568910] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.568947] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4377.569757] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4377.573623] Lustre: Skipped 1 previous similar message [ 4378.774832] Lustre: server umount lustre-MDT0000 complete [ 4380.030261] LustreError: 269629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195123 with bad export cookie 7456078420184164688 [ 4380.033638] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4380.036038] LustreError: 269629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4380.146024] Lustre: server umount lustre-MDT0001 complete [ 4384.016238] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4385.119732] Key type lgssc unregistered [ 4385.246346] LNet: 273229:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4385.248737] LNetError: 273229:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4385.256290] LNet: Removed LNI 192.168.202.133@tcp [ 4385.562266] Key type .llcrypt unregistered [ 4385.563485] Key type ._llcrypt unregistered [ 4392.490473] Key type ._llcrypt registered [ 4392.491515] Key type .llcrypt registered [ 4392.524596] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4392.897506] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4392.905182] alg: No test for adler32 (adler32-zlib) [ 4393.776540] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 4393.870877] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4395.455156] Key type lgssc registered [ 4395.834284] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4398.851567] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4398.857409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4414.303314] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4419.423340] LustreError: 274399:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.133@tcp: failed processing log, type 4: rc = -110 [ 4446.047182] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4448.021215] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4450.508683] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4451.296270] Lustre: server umount lustre-OST0000 complete [ 4457.126953] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4458.143207] Key type lgssc unregistered [ 4458.280350] LNet: 276031:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4458.282759] LNetError: 276031:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4458.291311] LNet: Removed LNI 192.168.202.133@tcp [ 4458.622161] Key type .llcrypt unregistered [ 4458.623409] Key type ._llcrypt unregistered [ 4472.594698] Key type ._llcrypt registered [ 4472.595750] Key type .llcrypt registered [ 4472.635656] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4472.908626] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4472.924125] alg: No test for adler32 (adler32-zlib) [ 4473.771511] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 4473.860473] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4475.439177] Key type lgssc registered [ 4475.808859] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4480.139462] loop0: detected capacity change from 0 to 204800000 [ 4480.205919] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4480.209652] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4480.335301] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4480.683856] loop0: detected capacity change from 0 to 204800000 [ 4480.786374] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4480.791720] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4480.906392] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4481.291067] loop0: detected capacity change from 0 to 204800000 [ 4481.352684] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4481.355939] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4481.398182] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4481.771985] loop0: detected capacity change from 0 to 204800000 [ 4481.835360] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4482.231463] loop0: detected capacity change from 0 to 204800000 [ 4482.241363] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4483.584828] loop0: detected capacity change from 0 to 204800000 [ 4483.593277] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4484.991315] loop0: detected capacity change from 0 to 204800000 [ 4485.007311] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4486.282853] loop0: detected capacity change from 0 to 204800000 [ 4486.298204] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4487.582506] loop0: detected capacity change from 0 to 204800000 [ 4487.613394] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4487.627871] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4487.682632] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4487.685757] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4487.689078] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4487.692254] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4487.695583] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4487.698692] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4487.701814] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4487.705058] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4487.708229] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4487.711468] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4487.715309] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4487.719129] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4487.723122] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4487.726869] Lustre: 279420:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4490.280686] Lustre: server umount MGS complete [ 4490.568518] loop0: detected capacity change from 0 to 204800000 [ 4490.594114] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4490.739670] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4490.754292] Lustre: lustre-MDT0000: Aborting client recovery [ 4491.047790] loop1: detected capacity change from 0 to 204800000 [ 4491.082633] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4491.156749] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4491.159887] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4491.163246] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4491.166775] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4491.169931] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4491.172977] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4491.176674] Lustre: 279854:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4491.183049] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4491.245897] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4491.271344] Lustre: lustre-MDT0001: Aborting client recovery [ 4491.580322] loop2: detected capacity change from 0 to 204800000 [ 4491.608546] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4491.661674] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4491.666491] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4491.669962] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4491.673913] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4491.677575] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4491.681385] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4491.684594] Lustre: 279936:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4491.741238] Lustre: Found index 0 for lustre-OST0000, updating log [ 4491.751739] Lustre: 279955:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4491.773441] Lustre: lustre-OST0000: Aborting recovery [ 4492.082091] loop3: detected capacity change from 0 to 204800000 [ 4492.113895] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4492.179447] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4492.182711] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4492.186121] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4492.189656] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4492.193251] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4492.196332] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4492.199478] Lustre: 280020:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4497.381658] LustreError: 279798:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4500.449334] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4500.449592] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4500.460803] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4500.463585] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4501.473237] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4501.473451] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4502.581198] LustreError: 279762:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 232us: rc = -2 [ 4516.831673] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4516.832936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4516.834883] Lustre: Skipped 2 previous similar messages [ 4516.837915] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4520.086567] Lustre: server umount lustre-MDT0000 complete [ 4520.369552] LustreError: 279861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195264 with bad export cookie 18059647828820366660 [ 4520.371832] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4520.375770] LustreError: 279861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4521.951600] LustreError: 279759:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4521.951671] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4521.958965] LustreError: 279759:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4521.959269] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4521.959275] Lustre: Skipped 3 previous similar messages [ 4521.964592] Lustre: Skipped 2 previous similar messages [ 4525.983433] LustreError: 279867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4525.983583] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4525.987967] LustreError: 279867:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4525.989638] Lustre: Skipped 2 previous similar messages [ 4526.574601] Lustre: server umount lustre-MDT0001 complete [ 4532.994774] Lustre: server umount lustre-OST0000 complete [ 4539.417476] Lustre: server umount lustre-OST0001 complete [ 4542.064875] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 09:21:25 (1786195285) [ 4551.253338] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 4554.299407] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4566.344985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4566.427752] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4566.437236] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4566.470640] Lustre: lustre-MDT0000: new disk, initializing [ 4566.490568] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4566.493642] Lustre: Skipped 2 previous similar messages [ 4566.498438] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4568.034639] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4572.170122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4572.224658] Lustre: 284081:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4572.255755] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4572.258056] Lustre: Skipped 1 previous similar message [ 4572.303273] Lustre: lustre-MDT0001: new disk, initializing [ 4572.344444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4572.360283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4572.364899] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4573.843042] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4577.783876] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4579.581421] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4581.901869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4582.041058] Lustre: lustre-OST0000: new disk, initializing [ 4582.043454] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4582.046858] Lustre: 286045:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4582.050143] Lustre: 286045:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 4582.090560] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4584.336299] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4587.513592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4587.517678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4587.549466] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4588.129827] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4590.537939] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4594.440237] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4599.013172] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4606.559972] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4648.929125] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4648.929911] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4648.931788] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4648.934761] Lustre: Skipped 1 previous similar message [ 4648.937890] Lustre: Skipped 1 previous similar message [ 4650.524329] Lustre: server umount lustre-OST0000 complete [ 4654.050629] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4654.052385] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4654.057857] Lustre: Skipped 1 previous similar message [ 4664.288214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4664.290519] Lustre: Skipped 4 previous similar messages [ 4666.335215] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4666.425155] Lustre: server umount lustre-MDT0000 complete [ 4667.632276] LustreError: 284072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195411 with bad export cookie 18059647828820369369 [ 4667.635542] LustreError: 284072:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4667.635665] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4667.752570] Lustre: server umount lustre-MDT0001 complete [ 4670.571797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4671.881822] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4673.484758] Lustre: server umount MGS complete [ 4677.395441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4677.620206] LustreError: 289859:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4677.627202] LustreError: 289872:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4677.656994] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4679.116083] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4681.941804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4683.298154] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4685.527148] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4687.172968] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4689.313886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4691.154605] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4693.422757] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4723.169439] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4723.171249] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4723.172538] LustreError: Skipped 1 previous similar message [ 4723.172837] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4723.176866] Lustre: Skipped 1 previous similar message [ 4728.465234] Lustre: server umount lustre-OST0000 complete [ 4733.409624] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4733.410228] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4733.413099] Lustre: Skipped 1 previous similar message [ 4735.843371] Lustre: server umount lustre-MDT0000 complete [ 4737.034667] LustreError: 292715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195480 with bad export cookie 18059647828820371308 [ 4737.040096] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4737.160513] Lustre: server umount lustre-MDT0001 complete [ 4740.684614] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4741.686434] Key type lgssc unregistered [ 4741.813464] LNet: 293823:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4741.815599] LNetError: 293823:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4741.827364] LNet: Removed LNI 192.168.202.133@tcp [ 4742.138703] Key type .llcrypt unregistered [ 4742.139683] Key type ._llcrypt unregistered [ 4753.581368] Key type ._llcrypt registered [ 4753.582269] Key type .llcrypt registered [ 4753.617471] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 4760.070434] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4760.472212] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4760.528668] alg: No test for adler32 (adler32-zlib) [ 4761.388409] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 4761.474200] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4763.055233] Key type lgssc registered [ 4763.394266] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4774.605939] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4774.611024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4775.696214] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4775.704989] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4775.739618] Lustre: lustre-MDT0000: new disk, initializing [ 4775.759849] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4775.767225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4777.039895] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4781.001341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4781.040156] Lustre: 297753:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4781.062476] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4781.064208] Lustre: Skipped 1 previous similar message [ 4781.099945] Lustre: lustre-MDT0001: new disk, initializing [ 4781.128885] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4781.141238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4781.145278] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4782.423828] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4786.058072] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4787.840995] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4790.088833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4790.200345] Lustre: lustre-OST0000: new disk, initializing [ 4790.201907] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4790.204350] Lustre: 299722:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4790.232137] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4792.115597] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4793.334809] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4793.337882] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4793.369745] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4795.759908] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4799.222605] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4801.757114] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4810.109920] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4819.713217] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4859.874178] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4859.876657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4859.877031] LustreError: Skipped 1 previous similar message [ 4859.878341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4859.880244] Lustre: Skipped 1 previous similar message [ 4864.995515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4864.997891] Lustre: Skipped 2 previous similar messages [ 4865.569288] Lustre: server umount lustre-OST0000 complete [ 4868.064275] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4868.067760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4868.074952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4870.112404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4870.113718] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4875.232243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4875.234487] Lustre: Skipped 2 previous similar messages [ 4881.375128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4881.440019] Lustre: server umount lustre-MDT0000 complete [ 4882.732500] LustreError: 297747:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195626 with bad export cookie 7600107929095453698 [ 4882.737824] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4882.882721] Lustre: server umount lustre-MDT0001 complete [ 4885.729510] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4887.061144] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4888.767783] Lustre: server umount MGS complete [ 4892.484143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4892.637777] LustreError: 303648:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4892.658976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4893.982527] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4896.859126] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4896.998771] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4898.421555] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4900.798399] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4902.671413] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4904.803270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4906.679260] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4909.040172] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4942.815883] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4942.819034] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4942.825513] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4942.828373] Lustre: Skipped 2 previous similar messages [ 4945.105614] Lustre: server umount lustre-OST0000 complete [ 4948.447836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4948.451531] LustreError: Skipped 1 previous similar message [ 4948.453727] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4948.458169] Lustre: Skipped 1 previous similar message [ 4952.470635] Lustre: server umount lustre-MDT0000 complete [ 4953.679240] LustreError: 303617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195697 with bad export cookie 7600107929095455609 [ 4953.681840] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4953.684941] LustreError: 303617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 4953.801125] Lustre: server umount lustre-MDT0001 complete [ 4957.545536] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4958.650862] Key type lgssc unregistered [ 4958.782455] LNet: 307600:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4958.784936] LNetError: 307600:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4958.791530] LNet: Removed LNI 192.168.202.133@tcp [ 4959.101626] Key type .llcrypt unregistered [ 4959.102680] Key type ._llcrypt unregistered [ 4965.727324] Key type ._llcrypt registered [ 4965.728342] Key type .llcrypt registered [ 4965.767772] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 4971.685052] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4972.116875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4972.123400] alg: No test for adler32 (adler32-zlib) [ 4972.970322] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 4973.061040] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4974.639102] Key type lgssc registered [ 4974.980096] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4982.334730] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4987.024549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4987.029480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4988.108991] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4988.118482] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4988.155205] Lustre: lustre-MDT0000: new disk, initializing [ 4988.177749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4988.183300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4989.450556] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4993.387582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4993.488082] Lustre: lustre-OST0000: new disk, initializing [ 4993.489585] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4993.491076] Lustre: Skipped 1 previous similar message [ 4993.492718] Lustre: 311619:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4993.513859] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4995.411189] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4996.288163] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4996.293126] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4996.301142] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 5165.537284] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5165.543839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5169.169636] Lustre: server umount lustre-MDT0000 complete [ 5170.333170] LustreError: 311618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195914 with bad export cookie 3337866134510725304 [ 5170.334678] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5170.336369] LustreError: 311618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5170.397675] Lustre: server umount lustre-OST0000 complete [ 5178.065663] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 5181.247897] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5192.799857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5192.889587] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5192.900444] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5192.933758] Lustre: lustre-MDT0000: new disk, initializing [ 5192.955744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5192.961421] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5194.315541] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5198.436334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5198.479764] Lustre: 315846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5198.509449] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5198.511116] Lustre: Skipped 1 previous similar message [ 5198.551570] Lustre: lustre-MDT0001: new disk, initializing [ 5198.586210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5198.598257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5198.601962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5199.915974] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5203.550050] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5205.376362] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5207.643192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5207.762918] Lustre: lustre-OST0000: new disk, initializing [ 5207.764526] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5207.766906] Lustre: 317814:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5207.797375] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5209.702254] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5213.175061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5213.177994] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5213.210398] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5213.412945] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5214.844272] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5214.921832] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5216.301953] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5216.385405] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5218.274625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5218.275633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5218.279124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5218.281729] Lustre: Skipped 1 previous similar message [ 5219.114630] Lustre: server umount lustre-OST0000 complete [ 5223.394044] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5223.395059] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5223.397610] Lustre: Skipped 2 previous similar messages [ 5226.523561] Lustre: server umount lustre-MDT0000 complete [ 5227.706556] LustreError: 316641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195971 with bad export cookie 3337866134513666081 [ 5227.709506] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5227.709664] LustreError: 316641:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5227.834622] Lustre: server umount lustre-MDT0001 complete [ 5228.931669] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5229.428726] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:32:53 (1786195973) [ 5231.490186] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5231.684824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5232.948184] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5235.653657] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5237.149358] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5239.367397] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5241.082757] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5243.238108] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5245.276632] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5247.719869] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5249.891744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5249.953229] Lustre: lustre-OST0001: new disk, initializing [ 5249.955197] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5249.957757] Lustre: 322960:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5249.985306] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5249.987577] Lustre: Skipped 2 previous similar messages [ 5251.830652] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5256.198443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5256.201310] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5256.216673] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5256.219969] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5257.250612] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5257.253879] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5258.274436] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5258.277190] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5260.322993] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5260.326211] LustreError: 323530:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5260.710867] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5260.779083] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5262.416502] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f5c600cc800.ost_server_uuid 50 [ 5262.869961] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f5c600cc800.ost_server_uuid in FULL state after 0 sec [ 5264.418548] LustreError: 323536:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5264.421377] LustreError: 323536:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5264.630170] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f5c600cc800.ost_server_uuid 50 [ 5265.156860] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9f5c600cc800.ost_server_uuid in FULL state after 0 sec [ 5266.471741] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5279.822489] Lustre: lustre-OST0001: Not available for connect from 192.168.202.33@tcp (stopping) [ 5279.824474] Lustre: Skipped 1 previous similar message [ 5281.763268] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5281.767840] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5281.770031] LustreError: Skipped 2 previous similar messages [ 5281.778587] Lustre: Skipped 1 previous similar message [ 5282.884793] Lustre: server umount lustre-OST0001 complete [ 5286.367755] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5286.369980] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5286.373750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5286.375555] Lustre: Skipped 2 previous similar messages [ 5286.885153] LustreError: 323854:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5286.892200] LustreError: 323854:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5290.636409] Lustre: server umount lustre-OST0000 complete [ 5292.001556] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5292.003892] LustreError: Skipped 1 previous similar message [ 5297.120681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5297.122874] Lustre: Skipped 4 previous similar messages [ 5306.335109] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5306.400186] Lustre: server umount lustre-MDT0000 complete [ 5307.539166] LustreError: 320104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196051 with bad export cookie 3337866134513666949 [ 5307.542430] LustreError: 320104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5307.542799] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5307.651830] Lustre: server umount lustre-MDT0001 complete [ 5310.901585] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 5311.812759] Key type lgssc unregistered [ 5311.927321] LNet: 325648:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5311.929527] LNetError: 325648:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5311.936307] LNet: Removed LNI 192.168.202.133@tcp [ 5312.211101] Key type .llcrypt unregistered [ 5312.212199] Key type ._llcrypt unregistered [ 5320.716733] Key type ._llcrypt registered [ 5320.717719] Key type .llcrypt registered [ 5320.750704] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5321.147939] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5321.157281] alg: No test for adler32 (adler32-zlib) [ 5321.996226] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 5322.075757] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5323.647125] Key type lgssc registered [ 5323.956379] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5326.333318] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5326.338705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5327.480784] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5328.639504] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5330.717586] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5332.690492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5332.805595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5333.960146] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5336.758389] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5336.860720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5337.889336] LustreError: 328907:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5337.896664] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5337.897215] LustreError: 328907:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5338.634075] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5340.913923] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5343.016264] LustreError: 328908:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5343.022040] LustreError: 328908:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5343.177550] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5343.248582] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5345.119820] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5347.415472] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5348.330179] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5349.864104] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5349.928423] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5425.126932] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5425.129112] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5425.131417] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5425.131704] LustreError: Skipped 1 previous similar message [ 5425.137969] Lustre: Skipped 1 previous similar message [ 5430.245761] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5430.250429] Lustre: Skipped 2 previous similar messages [ 5430.781533] Lustre: server umount lustre-OST0001 complete [ 5435.362176] LustreError: 328721:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5435.362797] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5435.363394] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5435.364478] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5435.367775] LustreError: 328721:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5435.369662] LustreError: Skipped 1 previous similar message [ 5438.339609] Lustre: server umount lustre-OST0000 complete [ 5440.481027] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5440.481430] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5440.482064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5440.482068] Lustre: Skipped 1 previous similar message [ 5440.484501] Lustre: Skipped 2 previous similar messages [ 5441.596694] Lustre: server umount lustre-MDT0000 complete [ 5443.264683] LustreError: 327026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196187 with bad export cookie 14953351331629795413 [ 5443.269486] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5443.271924] LustreError: 327026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5443.420654] Lustre: server umount lustre-MDT0001 complete [ 5447.409062] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 5448.271866] Key type lgssc unregistered [ 5448.375421] LNet: 331978:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5448.377464] LNetError: 331978:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5448.384310] LNet: Removed LNI 192.168.202.133@tcp [ 5448.660983] Key type .llcrypt unregistered [ 5448.661996] Key type ._llcrypt unregistered [ 5464.880405] Key type ._llcrypt registered [ 5464.881436] Key type .llcrypt registered [ 5464.921936] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5465.433276] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5465.450156] alg: No test for adler32 (adler32-zlib) [ 5466.299359] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 5466.378669] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5467.951097] Key type lgssc registered [ 5468.275661] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5471.024404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5471.030181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5472.187465] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5473.434371] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5476.001543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5476.130324] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5477.346190] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5478.175890] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5480.084038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5480.192820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5481.816418] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5482.274773] LustreError: 335729:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5482.278413] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5482.279145] LustreError: 335729:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5484.274170] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5484.344707] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5485.355106] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5485.975252] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5489.814929] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5497.309556] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5500.898484] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5500.900219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5500.901953] Lustre: Skipped 2 previous similar messages [ 5500.903597] Lustre: Skipped 1 previous similar message [ 5506.018136] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5506.020043] Lustre: Skipped 4 previous similar messages [ 5506.455318] Lustre: server umount lustre-MDT0000 complete [ 5507.547273] LustreError: 335378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196251 with bad export cookie 15994985625302660502 [ 5507.550386] LustreError: 335378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5507.551292] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5507.658458] Lustre: server umount lustre-MDT0001 complete [ 5518.334166] Lustre: server umount lustre-OST0000 complete [ 5529.619458] Lustre: server umount lustre-OST0001 complete [ 5534.236598] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5537.306824] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5537.432881] LustreError: 339500:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5537.448101] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5538.695266] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5541.235859] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5542.591950] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5543.476892] Lustre: 340640:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5545.451948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5545.561724] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5545.563551] Lustre: Skipped 1 previous similar message [ 5547.245082] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5547.619114] LustreError: 340994:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5547.622829] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5549.761615] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5551.561765] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5551.914577] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5554.549436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5555.717681] Lustre: 342508:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5556.703748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5556.705958] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5556.711010] Lustre: Skipped 1 previous similar message [ 5556.713655] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5562.338238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5562.340160] Lustre: Skipped 5 previous similar messages [ 5562.801424] Lustre: server umount lustre-MDT0000 complete [ 5564.097643] LustreError: 339529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196307 with bad export cookie 15994985625302662161 [ 5564.100885] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5564.102697] LustreError: 339529:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5564.235167] Lustre: server umount lustre-MDT0001 complete [ 5575.680938] Lustre: server umount lustre-OST0000 complete [ 5587.032399] Lustre: server umount lustre-OST0001 complete [ 5588.265725] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5588.880338] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5589.481343] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:38:53 (1786196333) [ 5597.029809] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5599.780750] loop0: detected capacity change from 0 to 1099511627776 [ 5599.920994] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5599.923848] *** VALIDATE xfs *** [ 5599.928067] XFS (loop0): Mounting V5 Filesystem [ 5599.936449] XFS (loop0): Ending clean mount [ 5600.319325] loop1: detected capacity change from 0 to 18691697672192 [ 5620.148869] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 5623.642970] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5635.197113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5635.274439] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5635.284331] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5635.318325] Lustre: lustre-MDT0000: new disk, initializing [ 5635.336912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5635.338737] Lustre: Skipped 1 previous similar message [ 5635.343146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5636.527050] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5640.372887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5640.410769] Lustre: 347621:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5640.431491] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5640.433144] Lustre: Skipped 1 previous similar message [ 5640.466939] Lustre: lustre-MDT0001: new disk, initializing [ 5640.506803] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5640.509681] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5641.693388] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5645.159715] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5646.926049] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5649.040782] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5649.136001] Lustre: lustre-OST0000: new disk, initializing [ 5649.137572] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5649.139911] Lustre: 349588:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5650.937839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5650.942159] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5650.942828] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5650.977868] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5654.516602] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5655.875672] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5655.935991] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5657.202237] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5657.266343] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5659.918177] Lustre: server umount lustre-OST0000 complete [ 5661.152377] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5661.153032] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5661.153253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5661.153259] Lustre: Skipped 1 previous similar message [ 5661.156216] Lustre: Skipped 4 previous similar messages [ 5670.391336] Lustre: server umount lustre-MDT0000 complete [ 5671.391884] LustreError: 347631:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5671.396259] LustreError: 347631:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5671.529209] LustreError: 348415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196415 with bad export cookie 15994985625302663694 [ 5671.531395] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5671.534257] LustreError: 348415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5682.188708] Lustre: server umount lustre-MDT0001 complete [ 5684.568071] XFS (loop0): Unmounting Filesystem [ 5685.962815] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:40:29 (1786196429) [ 5688.157710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5688.303517] LustreError: 352141:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5688.325606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5688.327477] Lustre: Skipped 2 previous similar messages [ 5689.573701] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5692.147808] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5693.477311] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5695.735583] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5697.517159] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5699.613685] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5701.496608] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5703.683551] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5723.615822] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5723.617709] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5723.621239] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5723.622724] Lustre: Skipped 3 previous similar messages [ 5728.148575] Lustre: server umount lustre-OST0000 complete [ 5729.760841] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5729.764189] Lustre: Skipped 1 previous similar message [ 5735.507956] Lustre: server umount lustre-MDT0000 complete [ 5736.645405] LustreError: 352124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196480 with bad export cookie 15994985625302664576 [ 5736.648779] LustreError: 352124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5736.649954] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5736.764707] Lustre: server umount lustre-MDT0001 complete [ 5740.045520] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 5740.955423] Key type lgssc unregistered [ 5741.074298] LNet: 355796:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5741.076651] LNetError: 355796:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5741.085300] LNet: Removed LNI 192.168.202.133@tcp [ 5741.352105] Key type .llcrypt unregistered [ 5741.353243] Key type ._llcrypt unregistered [ 5750.905013] Key type ._llcrypt registered [ 5750.905964] Key type .llcrypt registered [ 5750.945270] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5751.306833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5751.360561] alg: No test for adler32 (adler32-zlib) [ 5752.206429] Lustre: Lustre: Build Version: 2.17.56_51_geba2e8d [ 5752.291043] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5753.863092] Key type lgssc registered [ 5754.170301] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5756.408618] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5756.415869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5757.557413] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5758.806773] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5761.377602] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5761.489288] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5762.688111] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5764.911463] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5767.184184] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5770.401925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5770.593835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5773.142018] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5776.528518] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5792.224708] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5792.229654] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5792.237540] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5793.250524] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5793.252877] Lustre: Skipped 1 previous similar message [ 5793.253247] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5796.335302] Lustre: server umount lustre-MDT0001 complete [ 5797.445886] LustreError: 357363:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.33@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5797.452365] LustreError: 357363:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5798.372110] LustreError: 357339:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5798.383554] LustreError: 357339:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5800.420232] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5800.473968] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5800.477227] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5800.498253] Lustre: 357328:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5800.646736] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5802.150779] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5806.053808] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5807.589073] LustreError: 361070:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5807.591609] LustreError: 361070:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5807.594436] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5807.612243] LustreError: 361073:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -17 [ 5807.615516] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5807.615516] [ 6108.129348] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6108.129954] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6108.136518] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6108.139456] Lustre: Skipped 2 previous similar messages [ 6111.832603] Lustre: server umount lustre-MDT0001 complete [ 6113.250430] LustreError: 359837:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6113.259829] LustreError: 359837:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6116.506727] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6116.568858] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6116.571694] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6116.589585] Lustre: 357329:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6116.739732] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6118.350690] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 6121.964044] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6126.572715] LustreError: 373196:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6126.579047] LustreError: 373196:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6126.585535] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6126.589912] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6126.623754] LustreError: 373199:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6126.629483] LustreError: 373199:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6126.634052] LustreError: 373199:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -17 [ 6126.639507] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.133@tcp [ 6126.639507] [ 6422.095364] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [ 6422.097303] Lustre: Skipped 1 previous similar message [ 6424.034041] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6424.034100] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6424.037798] Lustre: Skipped 1 previous similar message [ 6427.205579] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [ 6427.207589] Lustre: Skipped 2 previous similar messages [ 6435.295126] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6435.382084] Lustre: server umount lustre-MDT0001 complete [ 6437.445430] LustreError: 357335:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.33@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6437.450517] LustreError: 357335:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6439.386213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6439.445224] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6439.448694] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6439.473470] Lustre: 357326:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6439.641701] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6441.581844] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 6445.033916] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6449.643686] LustreError: 385084:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6449.649685] LustreError: 385084:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6449.655290] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6449.659631] LustreError: 357331:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6449.697171] LustreError: 385087:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6449.708155] LustreError: 385087:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -114 [ 6449.714576] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6449.714576] [ 6757.344852] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6757.346062] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6757.347422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6757.349761] Lustre: Skipped 2 previous similar messages [ 6757.352452] Lustre: Skipped 7 previous similar messages [ 6762.388959] Lustre: server umount lustre-MDT0000 complete [ 6762.464926] LustreError: 357334:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6762.469229] LustreError: 357334:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 6763.485115] LustreError: 359180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786197507 with bad export cookie 6168284347607653204 [ 6763.488211] LustreError: 359180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6763.488651] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6767.584569] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6767.588148] Lustre: Skipped 1 previous similar message [ 6769.647042] Lustre: server umount lustre-MDT0001 complete [ 6770.856673] Lustre: server umount lustre-OST0000 complete [ 6773.538236] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 6776.224480] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 6787.087901] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 09:58:50 (1786197530) [ 6789.077184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6789.158438] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6789.167078] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6789.202486] Lustre: lustre-MDT0000: new disk, initializing [ 6789.221403] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6789.226444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6790.444048] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 6794.229359] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6794.268188] Lustre: 398791:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6794.288457] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6794.290071] Lustre: Skipped 1 previous similar message [ 6794.325172] Lustre: lustre-MDT0001: new disk, initializing [ 6794.354820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6794.365540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6794.370122] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6795.617851] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 6799.516067] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6801.799607] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6804.892832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6805.042707] Lustre: lustre-OST0000: new disk, initializing [ 6805.045808] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6805.051132] Lustre: 400757:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6805.100514] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6807.394080] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 6811.130589] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6811.134397] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6811.164175] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6811.456816] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6815.200163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6815.204392] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6815.211597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6815.214688] Lustre: Skipped 2 previous similar messages [ 6820.469612] Lustre: server umount lustre-MDT0000 complete [ 6821.348377] LustreError: 399601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6821.358564] LustreError: 399601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 6822.259911] LustreError: 398783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786197566 with bad export cookie 6168284347607655143 [ 6822.265095] LustreError: 399597:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6822.267177] LustreError: 398783:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6822.267341] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6822.397349] Lustre: server umount lustre-MDT0001 complete [ 6834.030482] Lustre: server umount lustre-OST0000 complete [ 6844.897059] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 6848.570929] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 6860.200669] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6572 sec ========== 10:00:03 (1786197603) [ 6860.756559] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:00:04 (1786197604) === [ 6862.350137] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:00:05 (1786197605) === [ 6879.717171] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 6881.209776] Key type lgssc unregistered [ 6881.369561] LNet: 407116:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6881.375378] LNetError: 407116:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6881.388559] LNet: Removed LNI 192.168.202.133@tcp [ 6881.908623] Key type .llcrypt unregistered [ 6881.911254] Key type ._llcrypt unregistered