[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 507756959 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001009] APIC: Switch to symmetric I/O mode setup [ 0.003084] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.007841] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009017] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011161] LSM: Security Framework initializing [ 0.012056] Yama: becoming mindful. [ 0.013044] SELinux: Initializing. [ 0.014064] *** VALIDATE selinux *** [ 0.022632] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027300] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028153] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029120] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030108] *** VALIDATE tmpfs *** [ 0.032399] *** VALIDATE proc *** [ 0.034106] *** VALIDATE cgroup *** [ 0.035009] *** VALIDATE cgroup2 *** [ 0.037131] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038154] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040034] Spectre V2 : User space: Vulnerable [ 0.041010] Speculative Store Bypass: Vulnerable [ 0.043497] debug: unmapping init [mem 0xffffffff8be59000-0xffffffff8be60fff] [ 0.045173] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046722] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047022] ... version: 2 [ 0.048014] ... bit width: 48 [ 0.049013] ... generic registers: 4 [ 0.050013] ... value mask: 0000ffffffffffff [ 0.051015] ... max period: 00007fffffffffff [ 0.052015] ... fixed-purpose events: 3 [ 0.053036] ... event mask: 000000070000000f [ 0.054292] rcu: Hierarchical SRCU implementation. [ 0.056442] smp: Bringing up secondary CPUs ... [ 0.057627] x86: Booting SMP configuration: [ 0.058026] .... node #0, CPUs: #1 #2 #3 [ 0.061646] smp: Brought up 1 node, 4 CPUs [ 0.063010] smpboot: Max logical packages: 1 [ 0.064018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.135690] node 0 deferred pages initialised in 69ms [ 0.138410] devtmpfs: initialized [ 0.140343] x86/mm: Memory block size: 128MB [ 0.142563] gcov: version magic: 0x41383552 [ 0.146189] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.149115] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.152308] pinctrl core: initialized pinctrl subsystem [ 0.153136] [ 0.153645] ************************************************************* [ 0.155011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.157011] ** ** [ 0.160012] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.161009] ** ** [ 0.163009] ** This means that this kernel is built to expose internal ** [ 0.166013] ** IOMMU data structures, which may compromise security on ** [ 0.168011] ** your system. ** [ 0.169009] ** ** [ 0.171011] ** If you see this message and you are not debugging the ** [ 0.173012] ** kernel, report this immediately to your vendor! ** [ 0.176011] ** ** [ 0.179011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.182011] ************************************************************* [ 0.185590] NET: Registered protocol family 16 [ 0.187343] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.190039] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.192054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.195051] cpuidle: using governor menu [ 0.196667] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.199592] PCI: Using configuration type 1 for base access [ 0.201133] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.208233] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.212079] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.217076] cryptd: max_cpu_qlen set to 1000 [ 0.220342] ACPI: Added _OSI(Module Device) [ 0.222012] ACPI: Added _OSI(Processor Device) [ 0.224016] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.226012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.231295] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.238470] ACPI: Interpreter enabled [ 0.240078] ACPI: PM: (supports S0 S3 S4 S5) [ 0.243030] ACPI: Using IOAPIC for interrupt routing [ 0.247128] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.251274] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.261436] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.264044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.267021] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.272067] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.274783] acpiphp: Slot [2] registered [ 0.276111] acpiphp: Slot [5] registered [ 0.277126] acpiphp: Slot [6] registered [ 0.278156] acpiphp: Slot [7] registered [ 0.280109] acpiphp: Slot [8] registered [ 0.281127] acpiphp: Slot [9] registered [ 0.282180] acpiphp: Slot [10] registered [ 0.284112] acpiphp: Slot [3] registered [ 0.285150] acpiphp: Slot [4] registered [ 0.286120] acpiphp: Slot [11] registered [ 0.288116] acpiphp: Slot [12] registered [ 0.290108] acpiphp: Slot [13] registered [ 0.292111] acpiphp: Slot [14] registered [ 0.293112] acpiphp: Slot [15] registered [ 0.294106] acpiphp: Slot [16] registered [ 0.295123] acpiphp: Slot [17] registered [ 0.297120] acpiphp: Slot [18] registered [ 0.298095] acpiphp: Slot [19] registered [ 0.300089] acpiphp: Slot [20] registered [ 0.301100] acpiphp: Slot [21] registered [ 0.303126] acpiphp: Slot [22] registered [ 0.304102] acpiphp: Slot [23] registered [ 0.306140] acpiphp: Slot [24] registered [ 0.308108] acpiphp: Slot [25] registered [ 0.310108] acpiphp: Slot [26] registered [ 0.311179] acpiphp: Slot [27] registered [ 0.313109] acpiphp: Slot [28] registered [ 0.315116] acpiphp: Slot [29] registered [ 0.317111] acpiphp: Slot [30] registered [ 0.318154] acpiphp: Slot [31] registered [ 0.320128] PCI host bridge to bus 0000:00 [ 0.321032] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.323035] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.326036] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.329033] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.332033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.335037] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.337219] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.341167] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.344477] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.355018] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.360080] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.362025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.364040] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.367029] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.370593] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.374500] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.380055] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.383075] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.387015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.403017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.408014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.414045] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.421018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.430014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.451020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.462000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.466017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.473015] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.487014] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.497411] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.502015] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.507015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.524015] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.537109] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.540000] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.546016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.560000] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.569600] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.576016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.588014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.606015] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.615871] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.627015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.638021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.661024] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.675672] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.678460] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.681701] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.685486] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.689318] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.696119] iommu: Default domain type: Passthrough [ 0.697536] SCSI subsystem initialized [ 0.699133] ACPI: bus type USB registered [ 0.700112] usbcore: registered new interface driver usbfs [ 0.703081] usbcore: registered new interface driver hub [ 0.705076] usbcore: registered new device driver usb [ 0.707161] pps_core: LinuxPPS API ver. 1 registered [ 0.708010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.711117] PTP clock support registered [ 0.713144] EDAC MC: Ver: 3.0.0 [ 0.714509] PCI: Using ACPI for IRQ routing [ 0.715410] NetLabel: Initializing [ 0.716000] NetLabel: domain hash size = 128 [ 0.718014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.719200] NetLabel: unlabeled traffic allowed by default [ 0.721244] vgaarb: loaded [ 0.722353] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.725019] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.730329] clocksource: Switched to clocksource kvm-clock [ 0.841941] VFS: Disk quotas dquot_6.6.0 [ 0.843757] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.846308] *** VALIDATE ramfs *** [ 0.847705] *** VALIDATE hugetlbfs *** [ 0.849403] pnp: PnP ACPI init [ 0.851915] pnp: PnP ACPI: found 6 devices [ 0.871116] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.873728] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.875389] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.877104] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.878853] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.880609] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.882693] NET: Registered protocol family 2 [ 0.884698] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.889228] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.894545] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.901307] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.904903] TCP: Hash tables configured (established 65536 bind 65536) [ 0.908081] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.911653] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.914265] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.916851] NET: Registered protocol family 1 [ 0.919453] RPC: Registered named UNIX socket transport module. [ 0.921262] RPC: Registered udp transport module. [ 0.922691] RPC: Registered tcp transport module. [ 0.924515] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.927384] NET: Registered protocol family 44 [ 0.929346] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.931338] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.933312] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.935714] PCI: CLS 0 bytes, default 64 [ 0.938577] Unpacking initramfs... [ 2.351260] debug: unmapping init [mem 0xffff8c6dfcc54000-0xffff8c6dfffbffff] [ 2.357382] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.359943] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.363234] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.866390] Initialise system trusted keyrings [ 2.868248] Key type blacklist registered [ 2.870232] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.880235] zbud: loaded [ 2.884954] *** VALIDATE nfs *** [ 2.886158] *** VALIDATE nfs4 *** [ 2.887750] pstore: using deflate compression [ 2.892932] Platform Keyring initialized [ 3.005093] NET: Registered protocol family 38 [ 3.006960] Key type asymmetric registered [ 3.008522] Asymmetric key parser 'x509' registered [ 3.010172] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.012905] io scheduler mq-deadline registered [ 3.014628] io scheduler kyber registered [ 3.016772] io scheduler bfq registered [ 3.018707] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.021873] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.024730] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.027691] ACPI: Power Button [PWRF] [ 3.032817] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.039425] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.061328] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.070354] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.090422] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.120692] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.147751] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.154709] Non-volatile memory driver v1.3 [ 3.156764] Linux agpgart interface v0.103 [ 3.185637] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.190296] vda: detected capacity change from 0 to 74698752 [ 3.206996] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.209983] vdb: detected capacity change from 0 to 1073741824 [ 3.225257] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.228454] vdc: detected capacity change from 0 to 2621440000 [ 3.242898] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.245834] vdd: detected capacity change from 0 to 2621440000 [ 3.262722] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.266355] vde: detected capacity change from 0 to 4294967296 [ 3.282992] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.286874] vdf: detected capacity change from 0 to 4294967296 [ 3.300579] libphy: Fixed MDIO Bus: probed [ 3.305876] usbcore: registered new interface driver usbserial_generic [ 3.310142] usbserial: USB Serial support registered for generic [ 3.313043] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.318165] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.320675] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.323567] mousedev: PS/2 mouse device common for all mice [ 3.326500] rtc_cmos 00:05: RTC can wake from S4 [ 3.329893] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.330183] rtc_cmos 00:05: registered as rtc0 [ 3.336295] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.340053] intel_pstate: CPU model not supported [ 3.343433] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.345907] hid: raw HID events driver (C) Jiri Kosina [ 3.350814] usbcore: registered new interface driver usbhid [ 3.350886] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.353224] usbhid: USB HID core driver [ 3.353435] drop_monitor: Initializing network drop monitor service [ 3.361801] Initializing XFRM netlink socket [ 3.363874] NET: Registered protocol family 10 [ 3.366796] Segment Routing with IPv6 [ 3.368253] NET: Registered protocol family 17 [ 3.371093] mpls_gso: MPLS GSO support [ 3.378819] RAS: Correctable Errors collector initialized. [ 3.381269] AVX version of gcm_enc/dec engaged. [ 3.384032] AES CTR mode by8 optimization enabled [ 3.463778] sched_clock: Marking stable (3463754074, 0)->(4438041836, -974287762) [ 3.466510] registered taskstats version 1 [ 3.468913] Loading compiled-in X.509 certificates [ 3.470677] zswap: loaded using pool lzo/zbud [ 3.494720] Key type big_key registered [ 3.505096] Key type encrypted registered [ 3.506822] ima: No TPM chip found, activating TPM-bypass! [ 3.509285] ima: Allocated hash algorithm: sha1 [ 3.511256] ima: No architecture policies found [ 3.513226] evm: Initialising EVM extended attributes: [ 3.515772] evm: security.selinux [ 3.517080] evm: security.ima [ 3.518256] evm: security.capability [ 3.519614] evm: HMAC attrs: 0x1 [ 3.522281] rtc_cmos 00:05: setting system clock to 2026-09-03 13:09:55 UTC (1788440995) [ 3.528393] debug: unmapping init [mem 0xffffffff8ce03000-0xffffffff8cffffff] [ 3.531635] debug: unmapping init [mem 0xffffffff8bb82000-0xffffffff8be58fff] [ 3.540082] Write protecting the kernel read-only data: 28672k [ 3.543456] debug: unmapping init [mem 0xffffffff8a203000-0xffffffff8a3fffff] [ 3.546649] debug: unmapping init [mem 0xffffffff8ab14000-0xffffffff8abfffff] [ 3.580560] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.589593] systemd[1]: Detected virtualization kvm. [ 3.592080] systemd[1]: Detected architecture x86-64. [ 3.594488] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.620888] systemd[1]: No hostname configured. [ 3.622852] systemd[1]: Set hostname to . [ 3.624953] random: systemd: uninitialized urandom read (16 bytes read) [ 3.627493] systemd[1]: Initializing machine ID from random generator. [ 3.666587] random: ln: uninitialized urandom read (6 bytes read) [ 3.771678] random: systemd: uninitialized urandom read (16 bytes read) [ 3.774386] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.779215] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.784085] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.492580] device-mapper: uevent: version 1.0.3 [ 4.495359] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.276517] virtio_net virtio0 ens2: renamed from eth0 [ 5.287738] random: fast init done [ 5.314977] scsi host0: ata_piix [ 5.331247] scsi host1: ata_piix [ 5.332638] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.334731] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.294672] dracut-initqueue[597]: RTNETLINK answers: File exists [ 10.027357] random: crng init done [ 10.030538] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.680869] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.814872] printk: systemd: 25 output lines suppressed due to ratelimiting [ 12.085927] SELinux: Disabled at runtime. [ 12.148340] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.157795] systemd[1]: Detected virtualization kvm. [ 12.159399] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.729333] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.732629] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.737031] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.744737] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.749717] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.757062] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.764331] systemd[1]: Starting Create list of required static device nodes for the current kernel... Starting Create list of required st…ce nodes for the current kernel... Mounting Kernel Debug File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. Starting Remount Root and Kernel File Systems... Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK [0[ 12.848599] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS m] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Control Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd File Systems. [ OK ] Reached target Slices. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 13.247081] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.591288] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.608935] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.737744] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.748431] EDAC sbridge: Ver: 1.1.2 [ 15.497307] Key type dns_resolver registered [ 15.852439] NFS: Registering the id_resolver key type [ 15.854354] Key type id_resolver registered [ 15.856040] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started irqbalance daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg654-server login: [ 42.298295] libcfs: loading out-of-tree module taints kernel. [ 42.318655] Key type ._llcrypt registered [ 42.319742] Key type .llcrypt registered [ 42.371587] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 54.520620] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 56.335933] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 56.344940] alg: No test for adler32 (adler32-zlib) [ 57.947409] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 59.090785] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 60.921682] Key type lgssc registered [ 61.577238] hrtimer: interrupt took 2403982 ns [ 63.157840] Lustre: Echo OBD driver; http://www.lustre.org/ [ 83.050020] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 128.232299] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 142.075609] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 142.116755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 143.383667] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 143.417511] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 143.503248] Lustre: lustre-MDT0000: new disk, initializing [ 143.649420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 143.699126] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 148.222322] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 160.502851] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 160.558473] Lustre: 6504:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 160.578402] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 160.581535] Lustre: Skipped 1 previous similar message [ 160.656204] Lustre: lustre-MDT0001: new disk, initializing [ 160.703542] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 160.748096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 160.768235] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 166.294778] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.618293] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 181.113266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 181.643714] Lustre: lustre-OST0000: new disk, initializing [ 181.647430] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 181.655273] Lustre: 8439:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 181.727192] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 185.913578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 185.933189] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 186.043649] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 188.754743] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 203.494945] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 203.688852] Lustre: lustre-OST0001: new disk, initializing [ 203.700808] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 203.711802] Lustre: 9512:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 203.791928] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 209.499422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 209.525961] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 209.704480] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 210.962404] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 223.632389] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 231.557277] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 238.052768] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing check_logdir /tmp/testlogs/ [ 243.280595] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing yml_node [ 247.919564] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 250.287920] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 252.721624] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 254.394982] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 09:14:03 EDT 2026 [ 271.297576] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 272.758972] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 288.738185] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 288.747461] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 288.771288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 291.809211] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 291.816251] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 291.823093] Lustre: Skipped 2 previous similar messages [ 291.845586] Lustre: Skipped 2 previous similar messages [ 294.388619] Lustre: server umount lustre-MDT0000 complete [ 297.915800] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441289 with bad export cookie 5586276973815348332 [ 297.921752] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 297.927349] LustreError: 6494:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 302.059971] LustreError: 6509:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 302.066148] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 302.074831] LustreError: 6509:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 302.081780] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 302.094399] Lustre: Skipped 1 previous similar message [ 302.114556] Lustre: Skipped 2 previous similar messages [ 307.171712] LustreError: 6511:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 307.172282] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 307.218459] Lustre: Skipped 1 previous similar message [ 308.195668] LustreError: 6511:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 308.214041] LustreError: 6511:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 312.289325] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 312.592447] Lustre: server umount lustre-MDT0001 complete [ 316.612341] Lustre: server umount lustre-OST0000 complete [ 320.361091] Lustre: server umount lustre-OST0001 complete [ 327.777929] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 337.176043] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 371.247683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 371.478723] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 371.521220] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 371.614417] Lustre: lustre-MDT0000: new disk, initializing [ 371.726463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 371.758442] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 376.075347] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 386.947663] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 387.112361] Lustre: 15069:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 387.169980] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 387.172293] Lustre: Skipped 1 previous similar message [ 387.298881] Lustre: lustre-MDT0001: new disk, initializing [ 387.416512] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 387.472837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 387.486741] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 393.029623] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 404.041911] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 411.566617] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 419.158792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 419.651301] Lustre: lustre-OST0000: new disk, initializing [ 419.655220] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 419.662173] Lustre: 17037:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 419.760704] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 421.448649] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 421.467346] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 421.590421] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 426.251834] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 436.363944] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 441.346661] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 441.587578] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 446.684791] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 446.889972] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 452.581412] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 452.584227] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 452.608250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 452.614713] Lustre: Skipped 3 previous similar messages [ 462.818568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 462.845169] Lustre: Skipped 3 previous similar messages [ 463.328186] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 463.530921] Lustre: server umount lustre-OST0000 complete [ 467.938204] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 467.955748] Lustre: Skipped 2 previous similar messages [ 481.248962] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 481.557851] Lustre: server umount lustre-MDT0000 complete [ 483.301728] LustreError: 16487:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 483.318597] LustreError: 16487:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 484.989074] LustreError: 17047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441476 with bad export cookie 5586276973815349865 [ 484.993044] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 484.998648] LustreError: 17047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 485.368291] Lustre: server umount lustre-MDT0001 complete [ 490.395198] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 09:17:59 (1788441479) [ 497.520228] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 497.879523] LustreError: 19305:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 497.940091] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 502.355813] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 507.363968] LustreError: 19305:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 507.389950] LustreError: 19305:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 511.648210] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 516.551167] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 524.114238] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 529.386343] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 536.056636] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 542.949952] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 551.211695] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 554.910601] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 563.171284] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 563.175858] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 563.187480] LustreError: Skipped 1 previous similar message [ 563.193887] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 563.216131] Lustre: Skipped 2 previous similar messages [ 563.223789] Lustre: Skipped 7 previous similar messages [ 567.442404] Lustre: server umount lustre-MDT0000 complete [ 568.290335] LustreError: 19305:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 568.326876] LustreError: 19305:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 571.509851] LustreError: 19285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441563 with bad export cookie 5586276973815350747 [ 571.523321] LustreError: 19285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 571.538256] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 572.121239] Lustre: server umount lustre-MDT0001 complete [ 586.615765] Lustre: server umount lustre-OST0000 complete [ 608.766520] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 619.478485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 619.894913] LustreError: 23687:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 619.982746] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 619.990287] Lustre: Skipped 2 previous similar messages [ 625.363559] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 633.401055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.961643] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 642.116122] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 649.646572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.240545] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 667.054417] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 667.241661] Lustre: lustre-OST0001: new disk, initializing [ 667.245669] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 667.255946] Lustre: 26001:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 668.611916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 668.630384] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 668.788535] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 674.545128] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 683.343056] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 686.916566] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 705.505637] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 705.514333] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 705.534598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 709.439532] Lustre: server umount lustre-MDT0000 complete [ 712.566678] LustreError: 23667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441704 with bad export cookie 5586276973815351587 [ 712.568623] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 712.590450] LustreError: 23667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 712.674667] LustreError: 23682:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 712.684354] LustreError: 23682:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 712.968169] Lustre: server umount lustre-MDT0001 complete [ 719.059340] Lustre: server umount lustre-OST0000 complete [ 722.963711] Lustre: server umount lustre-OST0001 complete [ 739.108243] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 749.731259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 750.290903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 750.294566] Lustre: Skipped 3 previous similar messages [ 754.351442] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 763.375749] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 768.560297] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 771.618394] Lustre: 30169:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 779.253379] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 785.924740] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 794.021948] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 800.532574] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 807.362374] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 810.789913] Lustre: 32047:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 820.197130] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 820.210729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 820.227173] Lustre: Skipped 5 previous similar messages [ 820.237195] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 820.247991] Lustre: Skipped 6 previous similar messages [ 822.711086] Lustre: server umount lustre-MDT0000 complete [ 826.128723] LustreError: 29773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441818 with bad export cookie 5586276973815352882 [ 826.140046] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 826.144465] LustreError: 29773:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 826.481075] Lustre: server umount lustre-MDT0001 complete [ 835.907290] Lustre: server umount lustre-OST0000 complete [ 839.110517] Lustre: server umount lustre-OST0001 complete [ 846.172964] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 09:23:55 (1788441835) [ 873.357765] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 883.898249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 884.203626] LustreError: 34846:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 884.214172] LustreError: 34846:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 888.847607] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 897.955983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 902.779156] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 905.672286] Lustre: 35996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 912.242994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 919.199695] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 927.914446] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 934.596530] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 942.299447] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 945.586339] Lustre: 37877:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 954.861698] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 954.872289] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 954.883899] Lustre: Skipped 6 previous similar messages [ 954.885188] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 954.890032] Lustre: Skipped 5 previous similar messages [ 956.697317] Lustre: server umount lustre-MDT0000 complete [ 960.571677] LustreError: 34827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441952 with bad export cookie 5586276973815353995 [ 960.577616] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 960.581136] LustreError: 34827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 961.044658] Lustre: server umount lustre-MDT0001 complete [ 975.531161] Lustre: server umount lustre-OST0000 complete [ 988.463685] Lustre: server umount lustre-OST0001 complete [ 997.062891] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 09:26:26 (1788441986) [ 1011.727515] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1019.863853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1020.264493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1020.269664] Lustre: Skipped 7 previous similar messages [ 1024.477057] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1032.681845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1037.571748] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1040.357898] Lustre: 41170:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1046.584272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1052.220562] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1060.259326] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1067.163149] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1075.152768] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1078.622633] Lustre: 43050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1079.436551] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1079.439853] Lustre: Skipped 1 previous similar message [ 1084.387736] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1084.410422] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1084.450076] Lustre: Skipped 2 previous similar messages [ 1088.052269] Lustre: server umount lustre-MDT0000 complete [ 1091.999875] LustreError: 42360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442083 with bad export cookie 5586276973815355101 [ 1092.006424] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1092.018226] LustreError: 42360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1106.403667] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1106.823178] Lustre: server umount lustre-MDT0001 complete [ 1115.260622] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1117.152973] LustreError: 3634:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8c6e45caf800 x1875316363887616/t0(0) o250->MGC192.168.206.154@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1122.135090] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1130.727131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1132.332564] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1135.984935] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1143.537495] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1145.062850] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1150.874893] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1152.680971] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1164.527967] Lustre: server umount lustre-MDT0000 complete [ 1168.356923] LustreError: 43923:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1168.366770] LustreError: 43923:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 30 previous similar messages [ 1168.863592] LustreError: 42360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442160 with bad export cookie 5586276973815356200 [ 1168.876415] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1168.892731] LustreError: 42360:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1169.449861] Lustre: server umount lustre-MDT0001 complete [ 1183.789528] Lustre: server umount lustre-OST0000 complete [ 1197.122642] Lustre: server umount lustre-OST0001 complete [ 1205.087611] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 09:29:54 (1788442194) [ 1219.969377] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1228.227730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1232.087544] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1240.078872] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1244.337312] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1247.047853] Lustre: 48807:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1253.379574] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1259.435677] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1267.294363] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1273.084798] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1279.732875] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1283.697204] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1295.795218] Lustre: server umount lustre-MDT0000 complete [ 1296.865239] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1299.556349] LustreError: 47636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442291 with bad export cookie 5586276973815357019 [ 1299.567608] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1299.573088] LustreError: 47636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1299.905378] Lustre: server umount lustre-MDT0001 complete [ 1313.149151] Lustre: server umount lustre-OST0000 complete [ 1326.993715] Lustre: server umount lustre-OST0001 complete [ 1336.322828] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 09:32:05 (1788442325) [ 1338.747856] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1340.435899] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 09:32:09 (1788442329) [ 1349.816952] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 1358.736509] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1381.340133] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1381.528323] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1381.616504] Lustre: lustre-MDT0000: new disk, initializing [ 1381.761213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1386.864935] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1396.323493] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1403.221272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1403.510889] Lustre: lustre-OST0000: new disk, initializing [ 1403.521052] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1403.525676] Lustre: Skipped 1 previous similar message [ 1403.532023] Lustre: 54883:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1405.209468] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1405.225339] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1405.271759] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1410.105764] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1419.519695] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1421.511678] Lustre: DEBUG MARKER: create test files [ 1451.492449] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1451.501026] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1451.523239] Lustre: Skipped 13 previous similar messages [ 1451.539586] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1451.541986] Lustre: Skipped 25 previous similar messages [ 1457.611728] Lustre: server umount lustre-OST0000 complete [ 1461.990606] Lustre: server umount lustre-MDT0000 complete [ 1474.123765] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 1477.066909] Key type lgssc unregistered [ 1477.351044] LNet: 56517:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1477.360305] LNetError: 56517:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1477.384130] LNet: Removed LNI 192.168.206.154@tcp [ 1478.323705] Key type .llcrypt unregistered [ 1478.326315] Key type ._llcrypt unregistered [ 1505.482436] Key type ._llcrypt registered [ 1505.485163] Key type .llcrypt registered [ 1505.593332] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1506.845989] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1506.889875] alg: No test for adler32 (adler32-zlib) [ 1507.965876] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1508.229495] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 1509.920165] Key type lgssc registered [ 1511.108607] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1518.446546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1518.483297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1519.989996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1524.147030] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1531.314672] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1538.691621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1539.027620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1544.189626] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1544.349710] Lustre: 59251:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1544.363191] Lustre: 59251:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1544.370957] Lustre: 59251:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1544.392709] Lustre: 59251:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1544.409131] Lustre: 59251:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1544.416534] Lustre: 59251:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1546.674902] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1554.228384] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1559.498711] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1564.945727] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1574.882650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1574.890490] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1574.900667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1579.294189] Lustre: server umount lustre-OST0000 complete [ 1583.771854] Lustre: server umount lustre-MDT0000 complete [ 1595.150815] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 1597.499836] Key type lgssc unregistered [ 1597.813988] LNet: 60859:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1597.824916] LNetError: 60859:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1598.890550] LNet: Removed LNI 192.168.206.154@tcp [ 1599.813194] Key type .llcrypt unregistered [ 1599.815071] Key type ._llcrypt unregistered [ 1629.145510] Key type ._llcrypt registered [ 1629.148218] Key type .llcrypt registered [ 1629.266842] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1630.483523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1630.611789] alg: No test for adler32 (adler32-zlib) [ 1631.662923] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1631.923022] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 1633.616161] Key type lgssc registered [ 1634.814115] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1643.060096] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1643.104524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1644.688563] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1649.189762] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1656.609420] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1664.337935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1664.693027] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1669.821962] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1669.951269] Lustre: 63696:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1669.957294] Lustre: 63696:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1669.964805] Lustre: 63696:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1669.973171] Lustre: 63696:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1669.983224] Lustre: 63696:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1669.992639] Lustre: 63696:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1671.678103] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1680.229446] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1685.285947] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1695.715216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1695.716276] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1695.730529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1699.846217] Lustre: server umount lustre-OST0000 complete [ 1704.064331] Lustre: server umount lustre-MDT0000 complete [ 1718.856967] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 1721.452320] Key type lgssc unregistered [ 1721.715629] LNet: 65257:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1721.725344] LNetError: 65257:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1721.746834] LNet: Removed LNI 192.168.206.154@tcp [ 1722.710629] Key type .llcrypt unregistered [ 1722.717159] Key type ._llcrypt unregistered [ 1741.609655] Key type ._llcrypt registered [ 1741.613457] Key type .llcrypt registered [ 1741.759137] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 1758.413858] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1760.010179] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1760.078970] alg: No test for adler32 (adler32-zlib) [ 1761.138585] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 1761.351518] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 1763.000348] Key type lgssc registered [ 1764.276752] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1801.494226] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1801.548263] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1802.886347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1802.929618] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1803.097267] Lustre: lustre-MDT0000: new disk, initializing [ 1803.160651] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1803.174716] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1808.117302] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1819.180592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1819.303869] Lustre: 68411:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1819.351732] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1819.356298] Lustre: Skipped 1 previous similar message [ 1819.427698] Lustre: lustre-MDT0001: new disk, initializing [ 1819.554558] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1819.597906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1819.610515] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1824.066728] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1834.426834] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1840.558842] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1847.895187] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1848.194697] Lustre: lustre-OST0000: new disk, initializing [ 1848.199897] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1848.206070] Lustre: 70383:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1848.326686] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1855.457956] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 1857.116095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1857.137949] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1857.261655] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1865.452730] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1871.356405] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1871.639279] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1876.873810] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1877.078106] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1883.111600] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1883.121852] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1883.127054] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1883.160268] Lustre: Skipped 1 previous similar message [ 1885.389448] Lustre: server umount lustre-OST0000 complete [ 1891.300059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1891.310670] LustreError: Skipped 1 previous similar message [ 1891.319259] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1891.340164] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1891.345907] Lustre: Skipped 1 previous similar message [ 1893.348870] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1893.349739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1893.368153] Lustre: Skipped 1 previous similar message [ 1895.755177] Lustre: server umount lustre-MDT0000 complete [ 1900.300022] LustreError: 68405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442892 with bad export cookie 5395085655011416675 [ 1900.316995] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1900.319156] LustreError: 68405:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1900.726646] Lustre: server umount lustre-MDT0001 complete [ 1909.947562] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 09:41:39 (1788442899) [ 1921.495777] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 1962.780696] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 1973.603237] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 2012.897146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2013.149575] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2013.181662] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2013.246658] Lustre: lustre-MDT0000: new disk, initializing [ 2013.347112] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2013.370571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2017.656572] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2027.520850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2027.654907] Lustre: 76404:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2027.723670] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2027.735199] Lustre: Skipped 1 previous similar message [ 2027.893749] Lustre: lustre-MDT0001: new disk, initializing [ 2027.980637] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2028.011049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2028.023181] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2032.490883] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2043.036834] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2049.238792] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2056.550890] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2056.808711] Lustre: lustre-OST0000: new disk, initializing [ 2056.815894] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2056.820783] Lustre: 78373:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2056.889424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2063.252358] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2065.460135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2065.477624] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2065.619365] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2073.820470] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2079.340786] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2079.615421] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2084.291697] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2084.548193] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2091.489943] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2091.508233] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2091.527266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2096.623348] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2096.636186] Lustre: Skipped 1 previous similar message [ 2101.216234] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2101.407479] Lustre: server umount lustre-OST0000 complete [ 2106.849674] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2106.852961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2106.877203] Lustre: Skipped 2 previous similar messages [ 2106.893018] Lustre: Skipped 2 previous similar messages [ 2120.160167] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2120.550056] Lustre: server umount lustre-MDT0000 complete [ 2121.185767] LustreError: 76412:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2121.203582] LustreError: 76412:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2124.383880] LustreError: 76396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443116 with bad export cookie 5395085655011417557 [ 2124.389756] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2124.402068] LustreError: 76396:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2124.709030] Lustre: server umount lustre-MDT0001 complete [ 2136.052488] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 09:45:25 (1788443125) [ 2142.270700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2142.837392] LustreError: 80786:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2142.912719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2147.501293] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2148.323922] LustreError: 80787:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2152.426311] LustreError: 80786:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2155.913685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2161.224186] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2168.939633] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2174.897193] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2182.542819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2182.863821] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2182.871477] Lustre: Skipped 1 previous similar message [ 2189.295703] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2196.694231] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2200.580902] Lustre: 83458:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 79bacb8c-d69d-416a-b4be-9162b6fcd9e6 at adminstrative request [ 2201.319557] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2201.358799] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2211.816635] Lustre: MGS: Client 79bacb8c-d69d-416a-b4be-9162b6fcd9e6 (at 0@lo) reconnecting [ 2211.829926] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4adf31d19fc2e147 to 0x4adf31d19fc2e3e7 [ 2211.843828] Lustre: MGC192.168.206.154@tcp: Connection restored to 0@lo (at 0@lo) [ 2237.619172] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2237.761225] Lustre: lustre-OST0001: new disk, initializing [ 2237.775388] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2237.783132] Lustre: 83851:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2243.556396] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2247.243053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2247.256764] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2247.368839] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2252.158659] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2257.891428] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2257.903959] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2257.905709] LustreError: Skipped 1 previous similar message [ 2257.906960] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2257.920823] Lustre: Skipped 1 previous similar message [ 2257.951096] Lustre: Skipped 5 previous similar messages [ 2260.870691] Lustre: server umount lustre-OST0001 complete [ 2263.024324] LustreError: 82609:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2263.038243] LustreError: 82609:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2265.061104] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2270.614084] Lustre: server umount lustre-OST0000 complete [ 2274.275529] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2274.284676] LustreError: Skipped 1 previous similar message [ 2274.290250] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2274.297923] Lustre: Skipped 2 previous similar messages [ 2280.497402] Lustre: server umount lustre-MDT0000 complete [ 2283.492263] LustreError: 80786:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2283.512738] LustreError: 80786:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2284.005305] LustreError: 83850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443275 with bad export cookie 5395085655011419111 [ 2284.006227] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2284.014354] LustreError: 83850:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2284.343692] Lustre: server umount lustre-MDT0001 complete [ 2293.417668] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 09:48:02 (1788443282) [ 2294.882464] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2296.895552] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 09:48:05 (1788443285) [ 2298.683768] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2300.431300] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 09:48:09 (1788443289) [ 2301.920451] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2303.971170] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 09:48:13 (1788443293) [ 2305.806818] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2307.716802] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 09:48:16 (1788443296) [ 2309.204952] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2314.989881] loop: module loaded [ 2315.000497] loop0: detected capacity change from 0 to 4294967296 [ 2315.036077] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2316.599661] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2320.303320] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2324.140110] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2350.779949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2350.844343] LustreError: 87977:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2350.869277] LustreError: 87977:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2350.881576] LustreError: 87977:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2350.895252] LustreError: 87977:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2371.172789] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 2381.528437] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 2417.565229] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2417.786727] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2417.809884] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2417.927197] Lustre: lustre-MDT0000: new disk, initializing [ 2418.016247] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2418.019057] Lustre: Skipped 1 previous similar message [ 2418.036974] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2422.193980] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2431.938546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2432.054829] Lustre: 90840:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2432.255027] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2436.782908] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2446.518633] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2452.083884] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2458.581615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2458.967820] Lustre: 92810:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2465.091610] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2467.381246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2467.450558] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2474.562784] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2479.468926] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2479.670680] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2484.306386] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2484.522252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2487.785484] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2487.794054] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2487.798907] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2487.807658] Lustre: Skipped 2 previous similar messages [ 2487.822139] Lustre: Skipped 7 previous similar messages [ 2492.516980] Lustre: server umount lustre-OST0000 complete [ 2510.304323] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2510.446988] Lustre: server umount lustre-MDT0000 complete [ 2512.360867] LustreError: 90849:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2512.395891] LustreError: 90849:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2514.035456] LustreError: 90835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443506 with bad export cookie 5395085655011419727 [ 2514.036596] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2514.048043] LustreError: 90835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2514.446984] Lustre: server umount lustre-MDT0001 complete [ 2523.247755] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:51:52 (1788443512) [ 2529.670920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2534.407985] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2543.187952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2548.864641] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2557.296392] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2563.532243] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2570.950516] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2571.376090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2571.386635] Lustre: Skipped 4 previous similar messages [ 2578.207285] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2586.308323] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2591.630229] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2591.810126] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2596.593849] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2596.826754] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2603.823352] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2603.980724] Lustre: lustre-OST0001: new disk, initializing [ 2603.983765] Lustre: Skipped 2 previous similar messages [ 2603.987651] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2603.991871] Lustre: Skipped 3 previous similar messages [ 2603.996469] Lustre: 98593:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2611.109704] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2613.894836] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2613.901939] Lustre: Skipped 2 previous similar messages [ 2613.908965] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2613.996084] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2621.593843] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2627.813103] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2628.216195] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2634.520952] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2634.953251] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2656.325759] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2683.883767] Lustre: Failing over lustre-MDT0000 [ 2684.617666] Lustre: server umount lustre-MDT0000 complete [ 2685.926689] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2685.940256] Lustre: Skipped 2 previous similar messages [ 2685.953279] LustreError: 95218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2685.983838] LustreError: 95218:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 2702.306274] Lustre: 66635:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788443677/real 1788443677] req@000000006f5fe9dc x1875318148446848/t0(0) o400->MGC192.168.206.154@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788443693 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2702.337861] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2710.841775] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2710.846273] LDISKFS-fs (dm-0): recovery complete [ 2710.857889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2712.547535] LustreError: 66634:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000002e5f4b64 x1875318148455552/t0(0) o250->MGC192.168.206.154@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2712.956364] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2714.285604] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2714.288561] Lustre: 95219:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2717.862989] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2718.193021] Lustre: 95222:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2718.204188] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2718.204915] Lustre: 95222:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2718.327039] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2718.649224] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2718.988911] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2718.997032] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2719.320155] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2719.327989] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2719.346464] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2719.357476] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2720.009868] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2720.017626] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2720.336121] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2720.345099] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2721.618481] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2721.636947] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2722.282624] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2722.294400] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2722.616193] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2722.629525] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2725.842869] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2725.863097] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2726.513969] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2726.534674] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2726.856157] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2726.864103] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2734.154617] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2734.171965] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2734.818220] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2734.839484] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2735.160192] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2735.172496] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2750.399929] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2750.414269] Lustre: 101448:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2751.061981] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2751.071342] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2751.392121] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2751.396822] LustreError: 101448:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2774.200144] Lustre: 101448:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2774.205620] Lustre: 101448:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2774.216784] Lustre: 101448:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 2fa1e923-c136-4110-9687-0c132f9648b2@192.168.206.54@tcp [ 2774.232458] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2774.244856] Lustre: 101448:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000fd72137a x1875318129365120/t0(8589934772) o35->2fa1e923-c136-4110-9687-0c132f9648b2@192.168.206.54@tcp:531/0 lens 392/0 e 0 to 0 dl 1788443776 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2774.288226] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2774.297345] Lustre: lustre-MDT0000: Denying connection for new client 2fa1e923-c136-4110-9687-0c132f9648b2 (at 192.168.206.54@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2774.327860] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2774.371293] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2774.373958] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2774.404889] Lustre: Skipped 2 previous similar messages [ 2774.436710] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2774.455972] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2790.882094] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2790.891747] LustreError: Skipped 1 previous similar message [ 2790.900805] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2790.909473] Lustre: Skipped 6 previous similar messages [ 2796.774242] Lustre: server umount lustre-OST0000 complete [ 2800.103245] LustreError: 97043:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2800.119968] LustreError: 97043:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 37 previous similar messages [ 2806.707993] Lustre: server umount lustre-OST0001 complete [ 2815.464823] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2815.477569] Lustre: Skipped 7 previous similar messages [ 2816.847355] Lustre: server umount lustre-MDT0000 complete [ 2820.872770] LustreError: 97065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443812 with bad export cookie 5395085655011470939 [ 2820.879127] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2820.886297] LustreError: 97065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2821.270354] Lustre: server umount lustre-MDT0001 complete [ 2829.821819] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:56:59 (1788443819) [ 2837.649778] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2837.656767] Lustre: *** cfs_fail_loc=197, val=0*** [ 2853.217275] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2858.336404] LustreError: 103421:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.154@tcp: failed processing log, type 4: rc = -110 [ 2863.482212] Lustre: *** cfs_fail_loc=197, val=0*** [ 2863.486614] Lustre: Skipped 65 previous similar messages [ 2884.064428] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2884.073392] Lustre: Skipped 2 previous similar messages [ 2890.873914] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2898.442397] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2900.653926] Lustre: server umount lustre-OST0000 complete [ 2909.073923] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:58:18 (1788443898) [ 2929.603374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2929.989747] LustreError: 105042:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2930.007695] LustreError: 105042:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2933.818686] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 2943.198931] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2947.222263] LustreError: 105073:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2947.231392] LustreError: 105073:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2947.244294] LustreError: 105073:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 2947.692566] Lustre: server umount lustre-MDT0000 complete [ 2983.009083] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 2994.358878] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3034.645892] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3034.875205] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3034.908467] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3034.961158] Lustre: lustre-MDT0000: new disk, initializing [ 3035.040549] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3039.502390] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3049.855035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3050.023651] Lustre: 109250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3050.359993] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3055.881849] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3066.387100] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3073.645283] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3080.472338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3080.721165] Lustre: 111231:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3086.878942] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3088.609273] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3096.053802] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3100.128186] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3100.379047] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3104.774950] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3105.032440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3109.347510] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3109.368045] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3109.369704] LustreError: Skipped 5 previous similar messages [ 3109.377728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3109.398196] Lustre: Skipped 1 previous similar message [ 3109.406140] Lustre: Skipped 9 previous similar messages [ 3113.468452] Lustre: server umount lustre-OST0000 complete [ 3132.384119] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3132.548309] Lustre: server umount lustre-MDT0000 complete [ 3136.403564] LustreError: 111230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444128 with bad export cookie 5395085655011481558 [ 3136.409159] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3136.416837] LustreError: 111230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3136.896285] Lustre: server umount lustre-MDT0001 complete [ 3145.109754] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 10:02:14 (1788444134) [ 3152.902235] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 3155.832278] Key type lgssc unregistered [ 3156.095719] LNet: 113721:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3156.106112] LNetError: 113721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3156.116504] LNet: Removed LNI 192.168.206.154@tcp [ 3156.849103] Key type .llcrypt unregistered [ 3156.851166] Key type ._llcrypt unregistered [ 3175.208396] Key type ._llcrypt registered [ 3175.212189] Key type .llcrypt registered [ 3175.287868] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 3189.893424] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3190.886589] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3190.964281] alg: No test for adler32 (adler32-zlib) [ 3192.015923] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3192.237841] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 3193.904451] Key type lgssc registered [ 3194.967260] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3235.205428] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3235.238747] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3236.421742] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3236.447820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3236.535356] Lustre: lustre-MDT0000: new disk, initializing [ 3236.604462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3236.618791] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3242.108300] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3254.334744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3254.769690] Lustre: lustre-OST0000: new disk, initializing [ 3254.774607] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3254.779746] Lustre: Skipped 1 previous similar message [ 3254.791775] Lustre: 117894:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3254.911306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3261.430077] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3262.639341] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3262.656147] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3262.689181] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3271.278819] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3271.461985] Lustre: lustre-OST0001: new disk, initializing [ 3271.465528] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3271.470129] Lustre: 118757:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3271.544122] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3278.208184] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3281.072049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3281.077029] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3281.104940] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3291.620751] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3291.624335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3291.637287] Lustre: Skipped 1 previous similar message [ 3293.859064] Lustre: server umount lustre-MDT0000 complete [ 3300.814383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3305.033605] LustreError: 117080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444297 with bad export cookie 6410730176331026734 [ 3305.041236] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3305.104878] Lustre: server umount lustre-OST0000 complete [ 3308.482873] Lustre: server umount lustre-OST0001 complete [ 3316.765746] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 10:05:06 (1788444306) [ 3343.350616] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 10:05:31 (1788444331) [ 3354.048751] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3375.890339] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 10:06:05 (1788444365) [ 3413.615681] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 3426.074365] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3463.988806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3464.228636] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3464.284115] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3464.373991] Lustre: lustre-MDT0000: new disk, initializing [ 3464.459064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3464.475601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3468.938906] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3480.306439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3480.397449] Lustre: 126510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3480.448435] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3480.452510] Lustre: Skipped 1 previous similar message [ 3480.556597] Lustre: lustre-MDT0001: new disk, initializing [ 3480.676627] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3480.711259] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3480.721158] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3485.602418] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3495.032686] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3500.897557] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3506.250086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3506.531197] Lustre: lustre-OST0000: new disk, initializing [ 3506.539276] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3506.550445] Lustre: 128478:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3506.647626] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3512.018916] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3516.058930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3516.075191] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3516.185414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3521.583274] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3526.533851] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3526.757927] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3531.505981] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3531.782500] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3536.868392] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3536.871445] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3536.882483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3536.882500] Lustre: Skipped 1 previous similar message [ 3536.888264] Lustre: Skipped 1 previous similar message [ 3540.148190] Lustre: server umount lustre-OST0000 complete [ 3547.107350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3547.108977] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3547.111472] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3547.111479] LustreError: Skipped 1 previous similar message [ 3547.114523] Lustre: Skipped 1 previous similar message [ 3547.142332] Lustre: Skipped 2 previous similar messages [ 3549.514779] Lustre: server umount lustre-MDT0000 complete [ 3552.561542] LustreError: 126502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444544 with bad export cookie 6410730176331027959 [ 3552.563156] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3552.570478] LustreError: 126502:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3552.907768] Lustre: server umount lustre-MDT0001 complete [ 3561.349860] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 10:09:10 (1788444550) [ 3567.280043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3567.707414] LustreError: 130891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3567.724675] LustreError: 130891:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3567.798850] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3571.092914] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3573.217907] LustreError: 130892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3577.325239] LustreError: 130891:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3579.138849] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3583.584789] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3593.673343] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3601.383234] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3609.064607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3609.261991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3609.275663] Lustre: Skipped 1 previous similar message [ 3615.028627] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3621.884680] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3630.140649] Lustre: *** cfs_fail_loc=159, val=0*** [ 3645.659165] Lustre: lustre-MDT0000: Client e59da77b-98dd-42c2-bf7f-f7afb144d6f7 (at 192.168.206.54@tcp) reconnecting [ 3647.018318] Lustre: *** cfs_fail_loc=159, val=0*** [ 3647.021146] Lustre: Skipped 6 previous similar messages [ 3662.543848] Lustre: lustre-MDT0000: Client e59da77b-98dd-42c2-bf7f-f7afb144d6f7 (at 192.168.206.54@tcp) reconnecting [ 3681.248866] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3681.260599] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3681.269341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3681.270347] Lustre: Skipped 1 previous similar message [ 3686.374790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3686.382451] Lustre: Skipped 2 previous similar messages [ 3686.636283] Lustre: server umount lustre-OST0000 complete [ 3691.489431] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3691.504623] Lustre: Skipped 1 previous similar message [ 3696.418375] Lustre: server umount lustre-MDT0000 complete [ 3696.616637] LustreError: 130892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3696.652188] LustreError: 130892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3700.412806] LustreError: 130873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444692 with bad export cookie 6410730176331028841 [ 3700.436387] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3700.437511] LustreError: 130873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3700.886211] Lustre: server umount lustre-MDT0001 complete [ 3711.452923] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 3714.704553] Key type lgssc unregistered [ 3715.032790] LNet: 134671:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3715.049734] LNetError: 134671:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3715.071857] LNet: Removed LNI 192.168.206.154@tcp [ 3716.116832] Key type .llcrypt unregistered [ 3716.119538] Key type ._llcrypt unregistered [ 3740.308470] Key type ._llcrypt registered [ 3740.315820] Key type .llcrypt registered [ 3740.452146] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3741.449601] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3741.476325] alg: No test for adler32 (adler32-zlib) [ 3742.485566] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3742.721033] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 3744.408203] Key type lgssc registered [ 3745.852369] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3753.542591] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3753.566635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3754.941726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3759.386691] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3764.204154] LustreError: 136088:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3764.222882] LustreError: 136088:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3767.816532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3768.167793] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3772.000585] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3779.629870] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3785.017120] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3791.607134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3791.919783] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3797.668646] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3799.126269] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3806.406428] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3815.591038] Lustre: *** cfs_fail_loc=159, val=0*** [ 3832.044623] Lustre: lustre-MDT0000: Client ff9ea5a4-f9f0-4d54-a2eb-ea5a5644e604 (at 192.168.206.54@tcp) reconnecting [ 3834.029025] Lustre: *** cfs_fail_loc=159, val=0*** [ 3834.038220] Lustre: Skipped 1 previous similar message [ 3849.423023] Lustre: lustre-MDT0001: Client ff9ea5a4-f9f0-4d54-a2eb-ea5a5644e604 (at 192.168.206.54@tcp) reconnecting [ 3850.899497] Lustre: *** cfs_fail_loc=159, val=0*** [ 3850.902394] Lustre: Skipped 4 previous similar messages [ 3866.333071] Lustre: lustre-MDT0001: Client ff9ea5a4-f9f0-4d54-a2eb-ea5a5644e604 (at 192.168.206.54@tcp) reconnecting [ 3871.035918] Lustre: 139123:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3873.670780] Lustre: *** cfs_fail_loc=159, val=0*** [ 3873.677224] Lustre: Skipped 1 previous similar message [ 3889.356496] Lustre: lustre-MDT0000: Client 722a79f5-6e17-424f-a47f-1c56274ec380 (at 192.168.206.54@tcp) reconnecting [ 3890.899850] Lustre: *** cfs_fail_loc=159, val=0*** [ 3890.902155] Lustre: Skipped 14 previous similar messages [ 3907.266223] Lustre: lustre-MDT0000: Client 722a79f5-6e17-424f-a47f-1c56274ec380 (at 192.168.206.54@tcp) reconnecting [ 3909.205662] Lustre: 139418:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3914.737737] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3914.740823] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3914.749708] LustreError: Skipped 1 previous similar message [ 3914.800175] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3916.786415] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3916.796757] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3916.798930] LustreError: Skipped 1 previous similar message [ 3916.798966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3916.836411] Lustre: Skipped 1 previous similar message [ 3920.510235] Lustre: server umount lustre-OST0000 complete [ 3927.009935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3927.010721] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3927.039578] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3927.046169] Lustre: Skipped 1 previous similar message [ 3930.465212] Lustre: server umount lustre-MDT0000 complete [ 3932.129804] LustreError: 136089:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3932.159974] LustreError: 136089:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3934.267459] LustreError: 136070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444926 with bad export cookie 9425246932254639252 [ 3934.276125] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3934.282346] LustreError: 136070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3934.892709] Lustre: server umount lustre-MDT0001 complete [ 3945.841917] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 3948.379483] Key type lgssc unregistered [ 3948.741680] LNet: 140474:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3948.752675] LNetError: 140474:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3948.776071] LNet: Removed LNI 192.168.206.154@tcp [ 3949.665112] Key type .llcrypt unregistered [ 3949.667840] Key type ._llcrypt unregistered [ 3973.911956] Key type ._llcrypt registered [ 3973.916474] Key type .llcrypt registered [ 3974.050864] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 3975.151208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3975.180093] alg: No test for adler32 (adler32-zlib) [ 3976.241980] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 3976.482677] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 3978.152157] Key type lgssc registered [ 3979.018847] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3985.003086] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3985.039020] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3986.410151] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3990.153814] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 3996.663371] LustreError: 141889:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3996.678060] LustreError: 141889:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3997.379272] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3997.768191] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4001.854480] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4008.501582] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4014.318712] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4021.449145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4021.799813] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4022.852307] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4032.074902] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4032.546890] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4041.815236] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4048.345341] Lustre: 144473:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4051.907544] Lustre: 144521:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4053.599452] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4054.464880] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4058.538906] Lustre: 144668:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4059.401162] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4060.355098] Lustre: 144766:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4067.811221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4067.816346] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4067.821804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4067.831911] Lustre: Skipped 1 previous similar message [ 4071.032419] Lustre: server umount lustre-OST0000 complete [ 4074.465549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4074.496000] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.510972] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.517907] Lustre: Skipped 1 previous similar message [ 4078.059862] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4078.065301] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4080.431522] Lustre: server umount lustre-MDT0000 complete [ 4083.172160] LustreError: 141886:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4083.200995] LustreError: 141886:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4084.270930] LustreError: 144570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445076 with bad export cookie 16677780234211912486 [ 4084.277442] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4084.278622] LustreError: 144570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4084.521169] Lustre: server umount lustre-MDT0001 complete [ 4094.463915] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 4096.988757] Key type lgssc unregistered [ 4097.308408] LNet: 145830:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4097.319851] LNetError: 145830:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4097.343244] LNet: Removed LNI 192.168.206.154@tcp [ 4098.073701] Key type .llcrypt unregistered [ 4098.076759] Key type ._llcrypt unregistered [ 4125.462109] Key type ._llcrypt registered [ 4125.471384] Key type .llcrypt registered [ 4125.619558] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4126.999608] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4127.186462] alg: No test for adler32 (adler32-zlib) [ 4128.211186] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4128.527838] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 4130.200154] Key type lgssc registered [ 4131.285741] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4138.804589] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4138.834087] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4140.344883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4144.127388] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4150.754262] LustreError: 147244:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4150.792478] LustreError: 147244:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4152.439595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4152.950322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4158.047367] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4165.091727] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4170.065405] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4176.735869] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4177.127279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4178.183938] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4184.712835] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4185.365357] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4194.502390] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4200.227383] Lustre: *** cfs_fail_loc=159, val=0*** [ 4216.537939] Lustre: lustre-MDT0000: Client 1f699879-9862-4df0-9632-476524df8242 (at 192.168.206.54@tcp) reconnecting [ 4221.413635] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4221.413855] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4221.426108] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4221.436586] Lustre: Skipped 1 previous similar message [ 4223.502756] Lustre: server umount lustre-OST0000 complete [ 4229.601024] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4229.604838] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4229.610742] Lustre: Skipped 1 previous similar message [ 4229.613687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4231.652476] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4231.655172] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4231.675771] Lustre: Skipped 1 previous similar message [ 4236.770450] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4236.779892] Lustre: Skipped 1 previous similar message [ 4241.377056] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4241.778716] Lustre: server umount lustre-MDT0000 complete [ 4241.906466] LustreError: 149850:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4241.948522] LustreError: 149850:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4246.780790] LustreError: 150154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445238 with bad export cookie 5352143486528212127 [ 4246.790627] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4246.798030] LustreError: 150154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4247.095184] Lustre: server umount lustre-MDT0001 complete [ 4263.496383] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 4266.232484] Key type lgssc unregistered [ 4266.660419] LNet: 150911:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4266.668683] LNetError: 150911:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4266.687674] LNet: Removed LNI 192.168.206.154@tcp [ 4268.129642] Key type .llcrypt unregistered [ 4268.131600] Key type ._llcrypt unregistered [ 4290.463326] Key type ._llcrypt registered [ 4290.470550] Key type .llcrypt registered [ 4290.579185] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4291.512424] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4291.559447] alg: No test for adler32 (adler32-zlib) [ 4292.651260] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4292.895524] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 4294.592195] Key type lgssc registered [ 4295.629312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4302.950526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4302.975259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4304.377162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4308.822874] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4314.593277] LustreError: 152322:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4314.613862] LustreError: 152322:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4316.724906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4317.006033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4320.846607] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4326.692264] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4331.277128] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4336.267068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4336.512725] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4341.130364] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4341.611942] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4346.246618] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4349.804674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4369.381081] Lustre: lustre-MDT0001: evicting 1a093b6d-3c88-4691-83ff-5d2bb3d4e119 (at 192.168.206.54@tcp) by administrative request [ 4369.386244] Lustre: Skipped 1 previous similar message [ 4398.049838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4398.066676] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4398.091024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4401.135241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4401.139371] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4401.172900] Lustre: Skipped 1 previous similar message [ 4403.783815] Lustre: server umount lustre-OST0000 complete [ 4409.314647] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4409.322718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4409.333837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4411.363551] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4411.364989] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4411.403409] Lustre: Skipped 1 previous similar message [ 4413.752272] Lustre: server umount lustre-MDT0000 complete [ 4416.482719] LustreError: 155437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4416.493911] LustreError: 155437:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4417.579993] LustreError: 152303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445409 with bad export cookie 9879757696814472136 [ 4417.581300] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4417.588946] LustreError: 152303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4417.949405] Lustre: server umount lustre-MDT0001 complete [ 4434.826896] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 4439.464555] Key type lgssc unregistered [ 4439.818875] LNet: 156413:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4439.826043] LNetError: 156413:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4439.852418] LNet: Removed LNI 192.168.206.154@tcp [ 4441.716481] Key type .llcrypt unregistered [ 4441.719826] Key type ._llcrypt unregistered [ 4522.367665] Key type ._llcrypt registered [ 4522.372436] Key type .llcrypt registered [ 4522.496853] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4523.513211] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4523.542365] alg: No test for adler32 (adler32-zlib) [ 4524.670350] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4524.930614] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 4526.704135] Key type lgssc registered [ 4528.007095] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4536.497383] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4536.513128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4538.162659] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4543.769950] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4548.578688] LustreError: 158904:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4548.617291] LustreError: 158904:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4553.381408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4553.825229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4559.027151] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4567.657858] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4573.972316] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4581.222966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4581.629035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4583.708214] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4587.762362] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4595.816786] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4599.035941] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4609.002173] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4609.022139] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4609.044045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4609.523076] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4614.050966] Lustre: server umount lustre-OST0000 complete [ 4619.746622] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4619.768486] Lustre: Skipped 1 previous similar message [ 4619.779969] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4619.793513] Lustre: Skipped 2 previous similar messages [ 4624.866234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4624.872722] Lustre: Skipped 1 previous similar message [ 4625.509200] Lustre: server umount lustre-MDT0000 complete [ 4629.833234] LustreError: 158884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445621 with bad export cookie 3807470077660683853 [ 4629.849276] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4629.855376] LustreError: 158884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4630.223146] Lustre: server umount lustre-MDT0001 complete [ 4643.130977] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 4646.227738] Key type lgssc unregistered [ 4646.546236] LNet: 162449:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4646.550728] LNetError: 162449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4646.569621] LNet: Removed LNI 192.168.206.154@tcp [ 4647.372361] Key type .llcrypt unregistered [ 4647.377029] Key type ._llcrypt unregistered [ 4675.882484] Key type ._llcrypt registered [ 4675.888339] Key type .llcrypt registered [ 4676.070184] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4678.032464] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4678.071297] alg: No test for adler32 (adler32-zlib) [ 4679.583051] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4680.053384] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 4681.960605] Key type lgssc registered [ 4683.546115] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4700.368957] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 10:28:09 (1788445689) [ 4719.154332] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 4729.872296] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4769.659924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4769.679973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4770.870929] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4770.979741] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4771.051792] Lustre: lustre-MDT0000: new disk, initializing [ 4771.130977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4771.147669] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4775.387861] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4785.440370] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4785.572967] Lustre: 166716:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4785.647467] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4785.654407] Lustre: Skipped 1 previous similar message [ 4785.727534] Lustre: lustre-MDT0001: new disk, initializing [ 4785.896361] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4785.971683] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4785.980859] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4790.466649] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4800.547282] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4806.973853] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4814.107366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4814.291624] Lustre: lustre-OST0000: new disk, initializing [ 4814.296300] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4814.308285] Lustre: 168686:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4814.406403] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4821.301511] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4821.659456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4821.674826] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4821.844742] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4830.117309] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4842.473914] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4842.478721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4842.483082] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4842.490986] Lustre: Skipped 1 previous similar message [ 4844.571725] Lustre: server umount lustre-OST0000 complete [ 4852.708594] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4852.711492] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4852.720340] LustreError: Skipped 1 previous similar message [ 4852.721179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4852.741691] Lustre: Skipped 1 previous similar message [ 4852.749491] Lustre: Skipped 1 previous similar message [ 4854.320823] Lustre: server umount lustre-MDT0000 complete [ 4857.675883] LustreError: 166708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445849 with bad export cookie 4041476992379674416 [ 4857.683240] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4857.961552] Lustre: server umount lustre-MDT0001 complete [ 4867.005940] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 4869.991776] Key type lgssc unregistered [ 4870.316352] LNet: 170590:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4870.326395] LNetError: 170590:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4870.337889] LNet: Removed LNI 192.168.206.154@tcp [ 4871.044270] Key type .llcrypt unregistered [ 4871.046327] Key type ._llcrypt unregistered [ 4895.792089] Key type ._llcrypt registered [ 4895.794076] Key type .llcrypt registered [ 4895.911926] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 4897.468937] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4897.504735] alg: No test for adler32 (adler32-zlib) [ 4898.668944] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 4899.013222] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 4900.739169] Key type lgssc registered [ 4902.249405] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4912.053512] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4912.069147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4913.417992] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4918.607869] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4923.877136] LustreError: 172007:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4923.892346] LustreError: 172007:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4926.914565] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4927.266502] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4930.876345] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4938.683583] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4944.342223] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4950.714776] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4950.984021] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4957.765461] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 4958.220674] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4966.203778] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4974.462526] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4974.481500] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4974.494613] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4974.500506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.518764] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4975.456861] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4975.480693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4975.492316] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4975.499939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.522737] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4976.797927] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4976.823689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4976.847096] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4976.866188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.876874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4977.671391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.500780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.510969] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4979.516981] Lustre: Skipped 2 previous similar messages [ 4979.537467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4979.549073] Lustre: Skipped 2 previous similar messages [ 4979.555222] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4979.566778] Lustre: Skipped 2 previous similar messages [ 4979.572299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.586982] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4979.596566] Lustre: Skipped 2 previous similar messages [ 4980.661419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.096856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.015554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.800899] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4983.807136] Lustre: Skipped 3 previous similar messages [ 4983.819919] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4983.831653] Lustre: Skipped 3 previous similar messages [ 4983.840741] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4983.844231] Lustre: Skipped 3 previous similar messages [ 4983.849456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.873714] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4983.878252] Lustre: Skipped 3 previous similar messages [ 4984.957900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.096221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.976901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.946545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.739622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.737959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.564501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.451102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.430783] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4992.439250] Lustre: Skipped 8 previous similar messages [ 4992.464562] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4992.477837] Lustre: Skipped 8 previous similar messages [ 4992.489792] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4992.494838] Lustre: Skipped 8 previous similar messages [ 4992.502359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.531634] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4992.547725] Lustre: Skipped 8 previous similar messages [ 4993.370535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.312917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.094572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.874174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.636374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.664642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.464679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.415857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.510942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.677724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.602174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.623862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.677065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.602712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.489270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.403421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.324343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.269609] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5009.276156] Lustre: Skipped 17 previous similar messages [ 5009.288946] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5009.297317] Lustre: Skipped 17 previous similar messages [ 5009.300520] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5009.305520] Lustre: Skipped 17 previous similar messages [ 5009.308532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.320115] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5009.325130] Lustre: Skipped 17 previous similar messages [ 5010.060288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.913900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.747252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.755641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.616511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.375284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.184736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.992704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.820245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.820860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.795392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.799912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.837350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.890341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.726629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.793636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.871594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.840038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.719613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.493183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.378882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.259663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.411472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.488942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.368111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.214069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.288275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.329866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.291843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.419795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.291952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.270897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.239787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.305739] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5041.313606] Lustre: Skipped 33 previous similar messages [ 5041.328700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5041.344379] Lustre: Skipped 33 previous similar messages [ 5041.348868] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5041.353771] Lustre: Skipped 33 previous similar messages [ 5041.358835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.367192] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5041.373381] Lustre: Skipped 33 previous similar messages [ 5042.330500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.208837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.152182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.125702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.993800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.991876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.802328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.716350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.520961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.360267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.262535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.187945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.964674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.737210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.484801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.154518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.873828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.655437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.432456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.228686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.020044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.692637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.438848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.276934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.159786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.921995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.635583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.403651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.261505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.289236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.093370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.865970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.647934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.480061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.393360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.249816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.944642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.676195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.437287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.376036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.175120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.193402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.125558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.160332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.078377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.001606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.870742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.898726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.926311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.997553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.878290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.686026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.580651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.621745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.548148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.598234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.404989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.259040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.179580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.171543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.091364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.868547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.757200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.694360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.651671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.564264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.259415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.032420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.884445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.928566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.716905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.511239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.312130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.248430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.147387] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5106.153619] Lustre: Skipped 74 previous similar messages [ 5106.179548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5106.196244] Lustre: Skipped 74 previous similar messages [ 5106.207069] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5106.218446] Lustre: Skipped 74 previous similar messages [ 5106.226319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.239752] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5106.241645] Lustre: Skipped 74 previous similar messages [ 5107.186458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.000335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.757392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.493992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.475641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.355427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.151628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.909361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.659525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.431965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.211384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.992181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.744960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.621278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.355662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.135852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.845330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.560760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.295234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.072632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.819773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.521563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.174391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.924852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.657583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.547825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.324959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.100100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.811129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.642595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.385781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.063584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.794700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.586569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.417052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.205726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.101884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.996057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.829874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.657520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.333987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.098641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.765326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.385186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.251575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.122450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.930569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.697856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.469480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.290702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.171857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.074504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.025897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.812679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.724379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.605062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.555092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.539543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.507315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.421659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.270204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.111161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.914531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.769661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.600931] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5158.617445] LustreError: 172027:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5158.619741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.496265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.496706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.352817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.314777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.245996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.260727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.226708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.216793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.024798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.966191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.891472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.952663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.961372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.067615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.981225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.821319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.801114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.709576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.411256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.479758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.405686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.269564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.005196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.900758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.656039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.501511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.487604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.481349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.325947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.298713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.236583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.319713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.242636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.229618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.237650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.190743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.861886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.816925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.692760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.777741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.746844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.691596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.576811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.532644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.401777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.288691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.187333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.068618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.897411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.870153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.716110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.717066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.931131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.185201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.192041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.234556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.119926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.951353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.755765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.712939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.663522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.672244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.535491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.634824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.669660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.621277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.450836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.439982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.250250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.539218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.620915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.951922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.374791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.542656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.700656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.830721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.911883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.694942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.497818] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5234.505413] Lustre: Skipped 143 previous similar messages [ 5234.518820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5234.529370] Lustre: Skipped 143 previous similar messages [ 5234.532846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5234.537984] Lustre: Skipped 143 previous similar messages [ 5234.540794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.548895] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5234.551268] Lustre: Skipped 143 previous similar messages [ 5235.571305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.598091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.465084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.431308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.771406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.784770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.683846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.436361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.273863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.306957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.431320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.270652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.083150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.782159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.599726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.458831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.435677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.234540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.226537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.223540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.163099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.101421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.964102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.897946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.854341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.682719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.595395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.523628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.491895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.375545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.126623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.965842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.061285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.132726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.018390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.846574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.678517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.578444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.439262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.274959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.180271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.089193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.897900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.714705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.374434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.081702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.831942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.591698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.410683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.330457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.167534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.016404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.787434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.636604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.442144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.296278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.069453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.028556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.886775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.717420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.511585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.517578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.352121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.265678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.224428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.161975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.020969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.980618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.978260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.894387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.999033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.888977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.833609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.622066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.476469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.345179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.208556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.970696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.857645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.939295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.972138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.862033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.730947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.719440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.522804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.592354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.528319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.652297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.871164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.109753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.324222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.210069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.198283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.951073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.850035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.832839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.651522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.530205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.526826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.373632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.311129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.159792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.993776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.904206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.831299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.693352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.589252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.394496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.352753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.274161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.273432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.103808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.008633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.100518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.899802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.668374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.484224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.388069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.332821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.398303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.296889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.289750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.068605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.924375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.652573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.423662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.117875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.892068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.721983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.534265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.336274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.199378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.080581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.842682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.602290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.423361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.301227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.215559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.286609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.058704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.896107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.790532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.705900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.536111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.411836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.470677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.333429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.305283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.192705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.155334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.987577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.972913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.938751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.948491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.929848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.976459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.822113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.773966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.631680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.454585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.248159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.114687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.093683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.118693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.991057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.997138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.951139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.092900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.942883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.917948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.896514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.880937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.808701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.684151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.557508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.454271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.236273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.978335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.961878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.835912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.862967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.806619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.052704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.237509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.394442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.550813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.342966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.350981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.203948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.402490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.309788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.245485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.154156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.135866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.272567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.312860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.309474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.298302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.400460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.476304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.631757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.780647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.825647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.822273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.557065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.452703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.288542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.488346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.618553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.860818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.846968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.029658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.966170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.815733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.832870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.758635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.679257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.579695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.523469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.447252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.497341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.446577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.436324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.412247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.175692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.031686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.846453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.711477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.480506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.406691] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5446.412943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.421889] LustreError: 172027:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5447.355764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.307617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.103038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.036564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.893722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.874494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.768334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.559715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.392460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.285568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.012294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.855549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.649831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.518267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.382911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.358220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.365249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.262456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.140593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.951915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.886277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.825290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.660616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.650635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.508461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.348425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.086581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.996386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.922037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.975597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.875437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.791277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.634234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.532728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.355062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.246707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.101088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.849764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.734608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.616905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.478396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.291193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.103814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.878263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.684246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.439498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.189222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.914800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.641055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.364975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.242968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.055260] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5491.059478] Lustre: Skipped 281 previous similar messages [ 5491.070791] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5491.079531] Lustre: Skipped 281 previous similar messages [ 5491.083080] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5491.087282] Lustre: Skipped 281 previous similar messages [ 5491.090548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.106797] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5491.111497] Lustre: Skipped 281 previous similar messages [ 5491.944984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.652447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.406494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.211801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.069385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.929936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.707500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.493723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.513493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.305340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.074280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.844242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.545926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.318462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.240852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.022211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.908201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.633497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.427088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.141594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.910125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.767249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.475063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.291992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.128120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.054857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.005552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.870830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.794564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.652727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.754947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.682723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.749960] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5518.767547] LustreError: 172027:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5518.776513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.797694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.886747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.934459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.952447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.971187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.980144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.842867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.733710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.631960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.509728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.437382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.334822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.260974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.053460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.843524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.727066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.873857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.905635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.726548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.657636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.525144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.349292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.268045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.172274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.191589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.049526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.938397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.633388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.396855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.235927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.969929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.742696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.479493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.318884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.055633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.903349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.767701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.647776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.614683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.535514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.433326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.353569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.254141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.156685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.036526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.964646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.043050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.854666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.784787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.610644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.542531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.509683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.412080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.249339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.096407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.880923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.716685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.564085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.346228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.220725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.251697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.250374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.229247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.080939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.068301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.855187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.618565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.386304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.229975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.087592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.979177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.848590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.878082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.788114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.817476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.815473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.843168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.609586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.444907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.330440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.258519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.005880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.861608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.581576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.471992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.487984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.445820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.476304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.441925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.411555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.440459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.324831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.205708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.129926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.019506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.037518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.954847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.854240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.855486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.672845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.654288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.450228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.278845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.219973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.218557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.101641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.078685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.007991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.985402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.902393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.862410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.788808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.584563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.341452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.229393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.163067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.987211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.759921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.591888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.436177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.318919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.009959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.884050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.945884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.645140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.575648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.391736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.172829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.080794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.886147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.695570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.459397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.183201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.969651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.773948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.686432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.482868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.432544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.315275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.098405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.837382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.598119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.407933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.187335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.885952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.718135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.462084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.158441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.864575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.605600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.577535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.409811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.209609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.011419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.845319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.615213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.246647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.031079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.765302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.434438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.269493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.064682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.743924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.800808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.689440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.509896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.305042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.114782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.031133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.043756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.060690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.206224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.149527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.054252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.064251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.875832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.715654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.609457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.673881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.504512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.399559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.232651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.991765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.872158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.643518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.620648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.451700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.357774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.295834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.284570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.155618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.029553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.992885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.797141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.653890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.626600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.534526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.530904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.575282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.510915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.478838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.376075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.409976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.211655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.187784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.230317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.141177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.168215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.989589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.901927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.774629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.763865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.592980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.590775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.568039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.604892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.482763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.344862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.216090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.246291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.264451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.201928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.319888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.310680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.160130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.247902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.253703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.212084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.161898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.233141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.362600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.365588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.535073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.529375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.601638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.676579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.606343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.579690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.497442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.399570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.340480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.481640] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5735.492843] LustreError: 172027:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5735.513783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.699457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.741546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.839777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.815044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.804243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.680728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.628649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.804093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.741520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.780159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.708167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.515579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.581576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.643154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.566240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.602102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.472579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.395366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.302944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.071829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.926116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.661065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.655385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.698762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.599595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.440855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.399369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.233567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.096066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.765866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.515948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.413895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.228491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.054401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.807678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.764690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.680416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.551964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.451170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.146796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.964838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.844714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.667645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.517643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.377606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.245242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.110946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.875819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.666747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.445216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.235770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.038428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.901212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.691965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.389624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.063507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.987418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.785387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.772422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.793361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.643833] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5789.650952] LustreError: 172027:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5789.655195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.459119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.253608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.106401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.934208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.658856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.390198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.134666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.963416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.696433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.554964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.394014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.220276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.019415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.895956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.709578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.558868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.299412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.013365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.747867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.521966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.183230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.919247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.749174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.597158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.454868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.353641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.169686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.119889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.978468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.887596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.817660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.790230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.656955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.489331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.318527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.257048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.370948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.416281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.097254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.940157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.721200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.695151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.595256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.445780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.390179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.430146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.409439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.336704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.400356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.237800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.151920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.221954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.174462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.141144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.076631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.158905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.119232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.112457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.133886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.150520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.307183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.328755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.396642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.129806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.350456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.323196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.879591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.744603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.762451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.663686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.504181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.198264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.091690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.874799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.606692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.249640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.959080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.677799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.591682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.562334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.347216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.121327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.006289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.797917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.735446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.579225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.462500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.337358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.267376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.230468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.102150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.943429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.216200] LustreError: 173829:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6262.223167] LustreError: 172027:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6937.058825] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6937.075254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6937.105785] Lustre: Skipped 428 previous similar messages [ 6937.117376] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6938.594537] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6938.597523] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6938.616572] Lustre: Skipped 1 previous similar message [ 6942.566294] Lustre: server umount lustre-OST0000 complete [ 6948.848731] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6953.146330] Lustre: server umount lustre-MDT0000 complete [ 6953.954884] LustreError: 172008:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6953.982832] LustreError: 172008:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6957.326951] LustreError: 220451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447949 with bad export cookie 5449511719609651072 [ 6957.329021] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6957.340602] LustreError: 220451:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6957.734814] Lustre: server umount lustre-MDT0001 complete [ 6969.798612] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 6972.923852] Key type lgssc unregistered [ 6973.231786] LNet: 227279:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6973.253150] LNetError: 227279:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6973.277639] LNet: Removed LNI 192.168.206.154@tcp [ 6974.177919] Key type .llcrypt unregistered [ 6974.179853] Key type ._llcrypt unregistered [ 7000.176640] Key type ._llcrypt registered [ 7000.181573] Key type .llcrypt registered [ 7000.291437] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7001.527220] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7001.578530] alg: No test for adler32 (adler32-zlib) [ 7002.677173] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7002.986933] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 7004.720176] Key type lgssc registered [ 7005.964926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7015.039990] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7015.072162] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7016.553368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7021.292356] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7026.149307] LustreError: 228694:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7026.189882] LustreError: 228694:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7031.289615] LustreError: 228695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7031.542415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7032.070442] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7038.159906] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7045.137173] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7051.661908] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7059.722475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7060.406959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7061.452366] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7067.591485] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7075.475886] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7080.342495] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7082.026357] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7082.037638] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7082.043537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7082.055244] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7082.057581] LustreError: 228714:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7087.082166] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7087.082913] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7087.111483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7087.825931] Lustre: lustre-OST0000: Not available for connect from 192.168.206.54@tcp (stopping) [ 7087.833929] Lustre: Skipped 1 previous similar message [ 7092.200517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7092.207620] Lustre: Skipped 1 previous similar message [ 7097.317275] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7097.325067] Lustre: Skipped 2 previous similar messages [ 7098.336266] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7098.923365] Lustre: server umount lustre-OST0000 complete [ 7102.104102] LustreError: 228714:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7135.086081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7135.429196] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7141.716402] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7149.970276] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7153.898868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.929743] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.930831] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7153.974683] Lustre: Skipped 1 previous similar message [ 7155.167345] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 7160.851874] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff990d47526800.ost_server_uuid 50 [ 7162.665947] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff990d47526800.ost_server_uuid in FULL state after 0 sec [ 7169.507604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7169.509489] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7169.518539] Lustre: Skipped 1 previous similar message [ 7169.526915] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7169.526923] Lustre: Skipped 1 previous similar message [ 7173.066715] Lustre: server umount lustre-OST0000 complete [ 7179.745766] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7179.749803] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7179.760373] Lustre: Skipped 1 previous similar message [ 7179.767630] Lustre: Skipped 1 previous similar message [ 7183.936098] Lustre: server umount lustre-MDT0000 complete [ 7184.867097] LustreError: 228694:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7184.897630] LustreError: 228694:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7188.046755] LustreError: 228676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448180 with bad export cookie 17245252155356015988 [ 7188.064118] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7188.067967] LustreError: 228676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7188.469463] Lustre: server umount lustre-MDT0001 complete [ 7198.537986] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 7201.458632] Key type lgssc unregistered [ 7201.698671] LNet: 233906:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7201.710488] LNetError: 233906:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7201.735427] LNet: Removed LNI 192.168.206.154@tcp [ 7202.678785] Key type .llcrypt unregistered [ 7202.680506] Key type ._llcrypt unregistered [ 7230.693696] Key type ._llcrypt registered [ 7230.695811] Key type .llcrypt registered [ 7230.784710] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7231.654053] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7231.718696] alg: No test for adler32 (adler32-zlib) [ 7232.868429] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 7233.111841] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 7234.760171] Key type lgssc registered [ 7235.928844] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7244.913120] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7244.961953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7246.308419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7250.377794] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7256.547476] LustreError: 235613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7256.567911] LustreError: 235613:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7258.895466] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7259.251964] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7263.997786] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7266.989939] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7274.070448] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7274.498925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7280.784522] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7289.633913] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7289.885196] Lustre: lustre-OST0001: new disk, initializing [ 7289.894616] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7289.911416] Lustre: 237923:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7290.035726] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7291.051989] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7296.537464] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7296.590224] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7296.600270] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7296.757649] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7307.988820] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7315.943541] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7332.845279] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7332.847298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7332.864165] Lustre: Skipped 3 previous similar messages [ 7332.874567] Lustre: Skipped 2 previous similar messages [ 7337.941274] Lustre: server umount lustre-MDT0000 complete [ 7337.969446] LustreError: 235613:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7338.001251] LustreError: 235613:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7342.064093] LustreError: 235595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448334 with bad export cookie 11065632330998825964 [ 7342.068389] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7342.077077] LustreError: 235595:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7342.334544] Lustre: server umount lustre-MDT0001 complete [ 7356.897935] Lustre: server umount lustre-OST0000 complete [ 7359.968181] Lustre: 234880:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448335/real 1788448335] req@ffff8c6e45e8a300 x1875323886071424/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448351 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7360.018772] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7361.673946] Lustre: server umount lustre-OST0001 complete [ 7386.008773] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7395.748244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7396.319269] LustreError: 241027:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7396.407974] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7401.320762] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7401.469028] LustreError: 241028:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7406.564697] LustreError: 241027:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7411.549780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7411.682741] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7411.691172] Lustre: Skipped 1 previous similar message [ 7411.904762] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7416.994294] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7420.048542] Lustre: 242169:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7428.025867] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7428.344779] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7435.096318] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7436.539702] LustreError: 242523:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7441.728722] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7443.935535] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7451.986670] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7460.430412] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7464.478895] Lustre: 244039:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7475.171681] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7475.184181] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7475.202169] Lustre: Skipped 3 previous similar messages [ 7475.209230] Lustre: Skipped 2 previous similar messages [ 7480.293047] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7480.301071] Lustre: Skipped 4 previous similar messages [ 7481.370483] Lustre: server umount mylustre-MDT0000 complete [ 7485.039843] LustreError: 241770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448477 with bad export cookie 11065632330998828736 [ 7485.052427] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7485.301298] Lustre: server umount mylustre-MDT0001 complete [ 7499.545711] Lustre: server umount mylustre-OST0000 complete [ 7501.795810] Lustre: 234879:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448477/real 1788448477] req@ffff8c6d5d1d8700 x1875323886131328/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448493 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7501.832700] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7503.679496] Lustre: server umount mylustre-OST0001 complete [ 7522.803176] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7532.326964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7532.695653] LustreError: 246347:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7532.714571] LustreError: 246347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7532.775498] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7532.782562] Lustre: Skipped 1 previous similar message [ 7537.017976] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7544.917415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7549.477490] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7552.468926] Lustre: 247490:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7558.980533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7565.702656] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7570.602836] LustreError: 247844:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7570.642266] LustreError: 247844:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7576.328680] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7577.769925] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7583.226761] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7591.211712] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7594.721676] Lustre: 249363:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7603.688515] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7603.705696] Lustre: Skipped 2 previous similar messages [ 7603.713585] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7603.726681] Lustre: Skipped 2 previous similar messages [ 7606.672822] Lustre: server umount tfs-MDT0000 complete [ 7610.553263] LustreError: 246322:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448602 with bad export cookie 11065632330998830402 [ 7610.555807] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7610.561926] LustreError: 246322:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7611.035548] Lustre: server umount tfs-MDT0001 complete [ 7625.519996] Lustre: server umount tfs-OST0000 complete [ 7629.280301] Lustre: 234879:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788448605/real 1788448605] req@ffff8c6d63e90000 x1875323886181760/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788448621 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7629.322742] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7629.330660] Lustre: Skipped 1 previous similar message [ 7630.426743] Lustre: server umount tfs-OST0001 complete [ 7654.918762] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7667.276174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7667.667568] LustreError: 251612:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7667.686109] LustreError: 251612:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 7667.746941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7667.751866] Lustre: Skipped 3 previous similar messages [ 7672.884540] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7682.827396] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7683.044040] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 7683.054352] Lustre: Skipped 1 previous similar message [ 7688.314448] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7691.402508] Lustre: 252753:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7699.471128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7707.119441] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7712.064050] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7717.556885] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7724.850430] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7732.705073] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7736.310577] Lustre: 254626:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7742.146412] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 11:18:51 (1788448731) [ 7758.827460] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7758.844286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7758.858462] Lustre: Skipped 3 previous similar messages [ 7764.059656] Lustre: server umount lustre-MDT0000 complete [ 7768.296374] LustreError: 251587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448760 with bad export cookie 11065632330998831837 [ 7768.299924] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7768.305171] LustreError: 251587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7774.721113] Lustre: server umount lustre-MDT0001 complete [ 7793.120124] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7793.286740] Lustre: server umount lustre-OST0000 complete [ 7803.573684] Lustre: server umount lustre-OST0001 complete [ 7810.636532] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 7821.822792] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 7861.277493] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7861.277493] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7861.277493] [ 7861.316648] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7861.625623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7861.668890] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7861.796395] Lustre: lustre-MDT0000: new disk, initializing [ 7861.942427] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7861.959271] Lustre: Skipped 3 previous similar messages [ 7861.996420] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7866.515775] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7877.214178] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7877.214178] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7877.214178] [ 7877.256874] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7877.407434] Lustre: 257811:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7877.470679] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7877.474132] Lustre: Skipped 1 previous similar message [ 7877.590760] Lustre: lustre-MDT0001: new disk, initializing [ 7877.770830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7877.787020] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7882.088368] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7892.403734] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7892.648766] Lustre: lustre-OST0000: new disk, initializing [ 7892.657404] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7892.664843] Lustre: 259399:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7894.579930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7894.594042] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7894.755557] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7899.010150] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7909.659890] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7909.873954] Lustre: lustre-OST0001: new disk, initializing [ 7909.878146] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7909.885927] Lustre: 260270:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7911.281433] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7911.295308] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7911.419451] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7916.568572] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 7927.688420] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 11:21:56 (1788448916) [ 7943.035276] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 11:22:12 (1788448932) [ 7961.571308] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7961.580480] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7961.596505] Lustre: Skipped 5 previous similar messages [ 7961.617227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7961.626189] Lustre: Skipped 5 previous similar messages [ 7972.320141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7972.472623] Lustre: server umount lustre-OST0000 complete [ 7972.838782] LustreError: 259383:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7972.856154] LustreError: 259383:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [ 7982.896565] Lustre: server umount lustre-MDT0000 complete [ 7986.610847] LustreError: 257805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448978 with bad export cookie 11065632330998833174 [ 7986.620420] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7986.633665] LustreError: 257805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7993.111317] Lustre: server umount lustre-MDT0001 complete [ 8003.622418] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 8076.093388] Lustre: server umount lustre-OST0001 complete [ 8082.919444] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 8098.591755] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 8134.461686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8134.836134] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8134.878080] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8134.990672] Lustre: lustre-MDT0000: new disk, initializing [ 8135.091964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8135.095900] Lustre: Skipped 3 previous similar messages [ 8135.106534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8140.299809] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8152.574653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8152.722161] Lustre: 265066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8152.780173] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8152.784425] Lustre: Skipped 1 previous similar message [ 8152.892173] Lustre: lustre-MDT0001: new disk, initializing [ 8153.031913] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8153.050795] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8158.205881] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8167.982562] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8173.707864] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8180.237420] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8180.532551] Lustre: 267035:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8182.108902] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8182.261946] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8186.262619] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8195.511265] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8202.738591] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8202.741567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8202.746675] LustreError: Skipped 2 previous similar messages [ 8202.758506] Lustre: Skipped 5 previous similar messages [ 8202.761474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8202.761483] Lustre: Skipped 9 previous similar messages [ 8207.636221] Lustre: server umount lustre-OST0000 complete [ 8225.248126] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8225.400114] Lustre: server umount lustre-MDT0000 complete [ 8228.919767] LustreError: 265861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449220 with bad export cookie 11065632330998834644 [ 8228.922557] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8228.935528] LustreError: 265861:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8229.312315] Lustre: server umount lustre-MDT0001 complete [ 8241.767354] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 8244.097837] Key type lgssc unregistered [ 8244.392829] LNet: 268802:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8244.419463] LNetError: 268802:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8244.439100] LNet: Removed LNI 192.168.206.154@tcp [ 8245.194436] Key type .llcrypt unregistered [ 8245.198816] Key type ._llcrypt unregistered [ 8268.842083] Key type ._llcrypt registered [ 8268.847732] Key type .llcrypt registered [ 8268.931502] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 8269.859911] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8269.925194] alg: No test for adler32 (adler32-zlib) [ 8271.056338] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8271.421648] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 8273.200212] Key type lgssc registered [ 8274.466727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8281.790286] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8281.807354] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8283.238801] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8286.396063] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8293.860662] LustreError: 270261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8293.877051] LustreError: 270261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8294.363376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8294.760605] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8299.786570] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8309.887325] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8316.620765] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8323.768803] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8324.091307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8328.173827] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8328.977108] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8334.787631] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8335.737619] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8338.401386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8338.418424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8338.428766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8339.504137] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8344.033803] Lustre: server umount lustre-OST0000 complete [ 8349.669255] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8349.673309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8349.680412] Lustre: Skipped 1 previous similar message [ 8349.698375] Lustre: Skipped 2 previous similar messages [ 8353.269844] Lustre: server umount lustre-MDT0000 complete [ 8354.799858] LustreError: 270261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8354.828149] LustreError: 270261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8356.232084] LustreError: 270241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449348 with bad export cookie 9084281562672068874 [ 8356.236751] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8356.248493] LustreError: 270241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8356.524639] Lustre: server umount lustre-MDT0001 complete [ 8365.438371] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 8367.928922] Key type lgssc unregistered [ 8368.174048] LNet: 273845:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8368.187065] LNetError: 273845:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8368.201804] LNet: Removed LNI 192.168.206.154@tcp [ 8368.976922] Key type .llcrypt unregistered [ 8368.981424] Key type ._llcrypt unregistered [ 8383.927350] Key type ._llcrypt registered [ 8383.930579] Key type .llcrypt registered [ 8384.044856] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 8385.060710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8385.093564] alg: No test for adler32 (adler32-zlib) [ 8386.105274] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8386.316621] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 8387.992164] Key type lgssc registered [ 8389.119727] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8397.638500] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8397.679697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8413.282564] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8418.400543] LustreError: 275040:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.154@tcp: failed processing log, type 4: rc = -110 [ 8445.088285] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8450.756352] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8458.208275] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8460.307505] Lustre: server umount lustre-OST0000 complete [ 8475.032734] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 8477.342605] Key type lgssc unregistered [ 8477.593527] LNet: 276674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8477.600328] LNetError: 276674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8477.609688] LNet: Removed LNI 192.168.206.154@tcp [ 8478.293179] Key type .llcrypt unregistered [ 8478.296911] Key type ._llcrypt unregistered [ 8520.548452] Key type ._llcrypt registered [ 8520.552053] Key type .llcrypt registered [ 8520.648229] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 8521.664890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8521.692418] alg: No test for adler32 (adler32-zlib) [ 8522.665477] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 8522.870465] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 8524.544161] Key type lgssc registered [ 8525.562412] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8538.767528] loop0: detected capacity change from 0 to 204800000 [ 8538.996132] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8539.009909] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8539.369214] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8540.362252] loop0: detected capacity change from 0 to 204800000 [ 8540.508358] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8540.516226] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8540.883514] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8541.898268] loop0: detected capacity change from 0 to 204800000 [ 8542.075663] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8542.083132] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8542.183627] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8543.321317] loop0: detected capacity change from 0 to 204800000 [ 8543.575502] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8544.733255] loop0: detected capacity change from 0 to 204800000 [ 8544.765736] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8548.552959] loop0: detected capacity change from 0 to 204800000 [ 8548.586880] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8552.251482] loop0: detected capacity change from 0 to 204800000 [ 8552.281967] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8556.094976] loop0: detected capacity change from 0 to 204800000 [ 8556.127215] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8559.620627] loop0: detected capacity change from 0 to 204800000 [ 8559.673598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8559.686835] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8559.772288] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8559.788375] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8559.806791] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8559.819666] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8559.830316] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8559.838550] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8559.851728] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8559.860181] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8559.871752] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8559.882946] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8559.890191] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8559.901837] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8559.908942] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8559.920883] Lustre: 280087:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8564.637101] Lustre: server umount MGS complete [ 8565.404214] loop0: detected capacity change from 0 to 204800000 [ 8565.451068] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8565.842622] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8565.880669] Lustre: lustre-MDT0000: Aborting client recovery [ 8566.542785] loop1: detected capacity change from 0 to 204800000 [ 8566.671441] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8566.748253] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8566.767787] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8566.783243] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8566.794708] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8566.806692] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8566.815056] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8566.825687] Lustre: 280519:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8566.839749] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8566.979938] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8567.018091] Lustre: lustre-MDT0001: Aborting client recovery [ 8568.021510] loop2: detected capacity change from 0 to 204800000 [ 8568.153571] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8568.307682] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8568.313232] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8568.344481] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8568.352695] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8568.361044] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8568.372679] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8568.377566] Lustre: 280601:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8568.437913] Lustre: Found index 0 for lustre-OST0000, updating log [ 8568.461207] Lustre: 280620:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8568.503115] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8568.516207] Lustre: lustre-OST0000: Aborting recovery [ 8569.435678] loop3: detected capacity change from 0 to 204800000 [ 8569.524907] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8569.690991] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8569.710252] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8569.738202] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8569.764817] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8569.784580] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8569.798638] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8569.810235] Lustre: 280685:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8569.835760] Lustre: Found index 1 for lustre-OST0001, updating log [ 8569.895850] Lustre: 280692:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8570.006495] Lustre: lustre-OST0001: Aborting recovery [ 8575.143087] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8575.161336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8575.179108] LustreError: 280463:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8575.191216] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8575.208123] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8576.164609] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8576.168498] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8579.091445] LustreError: 280430:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 5431us: rc = -2 [ 8579.597711] LustreError: 280430:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 722us: rc = -2 [ 8579.617682] LustreError: 280430:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 17 previous similar messages [ 8592.864815] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8592.882939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8592.897475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8596.962317] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8596.962927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8596.978103] Lustre: Skipped 2 previous similar messages [ 8596.982808] Lustre: Skipped 2 previous similar messages [ 8598.855084] Lustre: server umount lustre-MDT0000 complete [ 8599.616401] LustreError: 280413:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449591 with bad export cookie 9167388199688991986 [ 8599.645280] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8602.080784] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8602.081057] LustreError: 280971:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8602.084444] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8602.084454] Lustre: Skipped 1 previous similar message [ 8602.095845] Lustre: Skipped 1 previous similar message [ 8602.133757] LustreError: 280971:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8604.134868] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8604.134924] LustreError: 280428:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8604.151491] Lustre: Skipped 2 previous similar messages [ 8604.174680] LustreError: 280428:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8605.997792] Lustre: server umount lustre-MDT0001 complete [ 8613.131882] Lustre: server umount lustre-OST0000 complete [ 8620.497949] Lustre: server umount lustre-OST0001 complete [ 8631.722979] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 11:33:40 (1788449620) [ 8659.411960] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 8669.366159] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 8703.264533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8703.550959] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8703.576941] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8703.660800] Lustre: lustre-MDT0000: new disk, initializing [ 8703.768967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8703.775399] Lustre: Skipped 1 previous similar message [ 8703.806943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8708.170750] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8718.647457] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8718.778299] Lustre: 284762:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8718.843909] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8718.846352] Lustre: Skipped 1 previous similar message [ 8718.952230] Lustre: lustre-MDT0001: new disk, initializing [ 8719.138853] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8719.190276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8719.199933] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8722.980910] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8731.870658] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8737.695270] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8744.043418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8744.301660] Lustre: lustre-OST0000: new disk, initializing [ 8744.307735] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8744.317529] Lustre: 286729:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8744.427961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8749.903631] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8753.220552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8753.234657] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8753.300019] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8759.091631] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8765.146903] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8777.013892] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8781.150432] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8830.436607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8830.446156] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8830.463962] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8830.469645] Lustre: Skipped 1 previous similar message [ 8832.176828] Lustre: server umount lustre-OST0000 complete [ 8837.090577] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8837.097785] LustreError: Skipped 1 previous similar message [ 8837.105093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8837.117236] Lustre: Skipped 1 previous similar message [ 8840.674808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8840.683438] Lustre: Skipped 2 previous similar messages [ 8841.936990] Lustre: server umount lustre-MDT0000 complete [ 8845.690193] LustreError: 288339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449837 with bad export cookie 9167388199688994688 [ 8845.693456] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8845.699046] LustreError: 288339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8846.019484] Lustre: server umount lustre-MDT0001 complete [ 8854.211083] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8858.509839] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8863.231614] Lustre: server umount MGS complete [ 8874.439232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8874.960183] LustreError: 290544:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8874.988676] LustreError: 290557:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8875.042174] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8880.106087] LustreError: 290545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8880.148663] LustreError: 290545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8880.352411] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8884.262622] LustreError: 290544:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8889.607177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8894.823973] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8904.205646] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8911.174330] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8918.177160] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8918.584919] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8918.600304] Lustre: Skipped 1 previous similar message [ 8924.720592] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 8932.386688] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8965.604074] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8965.608637] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8965.612136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8965.620126] Lustre: Skipped 2 previous similar messages [ 8965.634814] LustreError: Skipped 1 previous similar message [ 8969.789748] Lustre: server umount lustre-OST0000 complete [ 8980.441266] Lustre: server umount lustre-MDT0000 complete [ 8980.962546] LustreError: 290545:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8980.989243] LustreError: 290545:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8984.206177] LustreError: 290526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449976 with bad export cookie 9167388199688996515 [ 8984.211055] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8984.220624] LustreError: 290526:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8984.604983] Lustre: server umount lustre-MDT0001 complete [ 8996.338272] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 8999.093889] Key type lgssc unregistered [ 8999.418884] LNet: 294497:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8999.431775] LNetError: 294497:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8999.460117] LNet: Removed LNI 192.168.206.154@tcp [ 9000.350090] Key type .llcrypt unregistered [ 9000.355290] Key type ._llcrypt unregistered [ 9040.915837] Key type ._llcrypt registered [ 9040.919815] Key type .llcrypt registered [ 9041.004552] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 9056.893668] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 9058.433750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9058.553876] alg: No test for adler32 (adler32-zlib) [ 9059.767888] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9060.035072] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 9061.784264] Key type lgssc registered [ 9063.013910] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9097.246264] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9097.278845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9098.564436] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9098.594410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9098.688177] Lustre: lustre-MDT0000: new disk, initializing [ 9098.787293] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9098.807924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9104.396251] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9115.184543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9115.387363] Lustre: 298455:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9115.480910] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9115.485529] Lustre: Skipped 1 previous similar message [ 9115.619892] Lustre: lustre-MDT0001: new disk, initializing [ 9115.772393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9115.840494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9115.850713] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9120.716244] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9130.567281] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9136.536474] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9143.859270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9144.193865] Lustre: lustre-OST0000: new disk, initializing [ 9144.202974] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9144.212354] Lustre: 300426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9144.313283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9150.244307] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9151.087265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9151.097282] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9151.241883] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9160.078356] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9168.143243] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9183.023860] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9186.762760] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9194.535382] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9243.617032] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9243.623828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9243.632696] LustreError: Skipped 1 previous similar message [ 9243.637119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9243.650192] Lustre: Skipped 1 previous similar message [ 9247.533796] Lustre: server umount lustre-OST0000 complete [ 9253.863155] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9253.872413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9253.880130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9253.896444] Lustre: Skipped 1 previous similar message [ 9257.770087] Lustre: server umount lustre-MDT0000 complete [ 9261.941870] LustreError: 301980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450253 with bad export cookie 5520616650483572867 [ 9261.953086] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9262.302548] Lustre: server umount lustre-MDT0001 complete [ 9272.195945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9277.466681] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9282.471458] Lustre: server umount MGS complete [ 9295.624772] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9296.440613] LustreError: 304347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9296.477353] LustreError: 304347:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9296.487981] LustreError: 304360:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9296.605237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9301.986642] LustreError: 304348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9302.315317] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9306.084179] LustreError: 304347:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9311.204368] LustreError: 304348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9311.786488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9312.263585] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9317.332253] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9326.207473] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9332.773703] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9340.386563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9340.897172] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9347.148275] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9354.123633] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9381.762979] Lustre: server umount lustre-OST0000 complete [ 9388.006328] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9388.018624] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9388.032619] Lustre: Skipped 2 previous similar messages [ 9388.044316] Lustre: Skipped 1 previous similar message [ 9393.121612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9393.131375] Lustre: Skipped 2 previous similar messages [ 9395.253571] Lustre: server umount lustre-MDT0000 complete [ 9398.116645] Lustre: 296680:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788450374/real 1788450374] req@00000000a3582716 x1875325801876864/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1788450390 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 9398.148453] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9398.165613] LustreError: 304344:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9398.178257] LustreError: 304344:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9398.688334] LustreError: 304329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450390 with bad export cookie 5520616650483574792 [ 9398.704587] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9398.709700] LustreError: 304329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 9399.073655] Lustre: server umount lustre-MDT0001 complete [ 9409.574390] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [ 9412.229512] Key type lgssc unregistered [ 9412.499778] LNet: 308298:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9412.508778] LNetError: 308298:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9412.519932] LNet: Removed LNI 192.168.206.154@tcp [ 9413.301415] Key type .llcrypt unregistered [ 9413.303888] Key type ._llcrypt unregistered [ 9435.029761] Key type ._llcrypt registered [ 9435.033558] Key type .llcrypt registered [ 9435.197826] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [ 9450.386803] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 9452.142887] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9452.230942] alg: No test for adler32 (adler32-zlib) [ 9453.374641] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [ 9453.641263] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [ 9455.336151] Key type lgssc registered [ 9456.335540] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9481.895440] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [ 9497.820291] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9497.840731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9499.148640] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9499.176699] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9499.289299] Lustre: lustre-MDT0000: new disk, initializing [ 9499.397121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9499.417558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9504.778625] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [ 9515.401514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9515.704765] Lustre: lustre-OST0000: new disk, initializing [ 9515.708483] Lustre: lustre-OST0000: Not available for connect from 0@lo (not set up) [ 9515.711211] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9515.731096] Lustre: Skipped 1 previous similar message [ 9515.735555] Lustre: 312354:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9515.811144] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9521.196243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9521.207331] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9521.251364] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9521.964438] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10468.326373] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10468.339228] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10468.686163] Lustre: server umount lustre-MDT0000 complete [10473.573578] LustreError: 311539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451465 with bad export cookie 13988625452993929852 [10473.593870] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10473.619831] LustreError: 311539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10473.774871] Lustre: server umount lustre-OST0000 complete [10505.209519] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [10516.014828] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [10554.970949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10555.292385] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10555.327844] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10555.461355] Lustre: lustre-MDT0000: new disk, initializing [10555.538775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10555.559650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10560.870930] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10571.367780] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10571.526597] Lustre: 316713:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10571.612121] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10571.617776] Lustre: Skipped 1 previous similar message [10571.779749] Lustre: lustre-MDT0001: new disk, initializing [10571.886833] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10571.943749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10571.957844] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10576.613558] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10586.193681] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10591.167920] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10597.149894] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10597.367280] Lustre: lustre-OST0000: new disk, initializing [10597.370923] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10597.384162] Lustre: 318684:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10597.466026] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10603.135858] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10606.181263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10606.205343] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10606.326542] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10612.159848] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10616.373685] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10616.660559] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10620.999978] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10621.188756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10627.054256] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10627.055065] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10627.058423] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10627.064653] LustreError: Skipped 1 previous similar message [10627.095210] Lustre: Skipped 1 previous similar message [10632.173501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10632.176872] Lustre: Skipped 2 previous similar messages [10637.281681] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10637.283090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10637.300551] Lustre: Skipped 1 previous similar message [10637.602448] Lustre: server umount lustre-OST0000 complete [10642.401727] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10642.412205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10642.423436] Lustre: Skipped 1 previous similar message [10647.550255] Lustre: server umount lustre-MDT0000 complete [10651.746586] LustreError: 316708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451643 with bad export cookie 13988625452996870678 [10651.751545] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10651.760544] LustreError: 316708:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10652.106432] Lustre: server umount lustre-MDT0001 complete [10655.425182] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10657.241756] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 12:07:26 (1788451646) [10663.940495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10664.337606] LustreError: 321000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10664.355363] LustreError: 321000:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10664.395078] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10668.546576] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10669.539817] LustreError: 321001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10674.662950] LustreError: 321000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10677.783941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10678.155718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10682.913212] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10690.921397] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10696.709089] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10703.905908] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10704.236288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10710.637699] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10717.764800] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10724.133774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10724.326758] Lustre: lustre-OST0001: new disk, initializing [10724.334677] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10724.340553] Lustre: 323835:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10724.439795] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10730.952866] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10732.687575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10732.701525] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10732.752146] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10732.758029] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10733.794372] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10733.813126] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10734.824236] LustreError: 324299:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10736.878678] LustreError: 324299:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10736.888414] LustreError: 324299:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10740.371389] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10740.687757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10741.091725] LustreError: 324299:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10741.107265] LustreError: 324299:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10747.183689] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff990d81f73000.ost_server_uuid 50 [10748.859873] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff990d81f73000.ost_server_uuid in FULL state after 0 sec [10749.283056] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10749.301652] LustreError: 324287:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10754.958397] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff990d81f73000.ost_server_uuid 50 [10756.210937] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff990d81f73000.ost_server_uuid in FULL state after 0 sec [10757.682502] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10771.661760] Lustre: lustre-OST0001: Not available for connect from 192.168.206.54@tcp (stopping) [10772.964458] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10772.969269] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10772.982679] Lustre: Skipped 1 previous similar message [10773.994080] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10777.269190] Lustre: server umount lustre-OST0001 complete [10779.109241] LustreError: 322821:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10779.122331] LustreError: 322821:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10784.232823] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10784.235112] LustreError: 324288:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10784.258950] Lustre: Skipped 2 previous similar messages [10784.292034] LustreError: 324288:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10789.282992] Lustre: server umount lustre-OST0000 complete [10794.477262] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10794.500790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10794.512987] Lustre: Skipped 7 previous similar messages [10799.803972] Lustre: server umount lustre-MDT0000 complete [10803.843126] LustreError: 320981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788451795 with bad export cookie 13988625452996871560 [10803.848689] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10803.859569] LustreError: 320981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10804.272956] Lustre: server umount lustre-MDT0001 complete [10815.069536] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [10817.601939] Key type lgssc unregistered [10817.890360] LNet: 326529:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10817.904833] LNetError: 326529:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10817.941519] LNet: Removed LNI 192.168.206.154@tcp [10818.804879] Key type .llcrypt unregistered [10818.806892] Key type ._llcrypt unregistered [10843.572792] Key type ._llcrypt registered [10843.574676] Key type .llcrypt registered [10843.674982] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [10844.479976] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10844.555258] alg: No test for adler32 (adler32-zlib) [10845.618822] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [10845.860485] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [10847.528185] Key type lgssc registered [10848.582606] Lustre: Echo OBD driver; http://www.lustre.org/ [10856.171396] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10856.189320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10857.694318] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10863.086256] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10867.171339] LustreError: 327951:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10867.185805] LustreError: 327951:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10869.947637] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10872.294838] LustreError: 327952:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10877.013771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10877.409143] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10882.421370] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10893.447928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10894.078608] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10901.457333] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10902.259657] LustreError: 329632:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10902.310275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10907.623199] LustreError: 330024:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10907.641615] LustreError: 330024:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10909.433585] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10912.743171] LustreError: 330023:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10912.785516] LustreError: 330023:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10917.420816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10917.744442] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10923.014759] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10924.284388] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [10932.098979] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10938.100464] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10938.374238] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11040.742679] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11040.749908] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11040.750691] Lustre: Skipped 1 previous similar message [11040.753299] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11045.858305] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11045.866755] Lustre: Skipped 2 previous similar messages [11046.155755] Lustre: server umount lustre-OST0001 complete [11050.988865] LustreError: 329633:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11051.029687] LustreError: 329633:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11056.104386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11056.115288] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11056.122726] LustreError: Skipped 1 previous similar message [11056.123698] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11056.152060] Lustre: Skipped 1 previous similar message [11058.984792] Lustre: server umount lustre-OST0000 complete [11066.348130] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11066.361476] Lustre: Skipped 1 previous similar message [11066.364925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11066.371883] Lustre: Skipped 2 previous similar messages [11068.675676] Lustre: server umount lustre-MDT0000 complete [11071.464672] LustreError: 331627:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11071.487616] LustreError: 331627:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11072.253919] LustreError: 331636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452064 with bad export cookie 17907896146171565758 [11072.259552] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11072.268161] LustreError: 331636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11072.690136] Lustre: server umount lustre-MDT0001 complete [11082.862692] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [11085.628736] Key type lgssc unregistered [11085.913409] LNet: 332897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11085.928044] LNetError: 332897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11085.950178] LNet: Removed LNI 192.168.206.154@tcp [11086.758152] Key type .llcrypt unregistered [11086.761733] Key type ._llcrypt unregistered [11128.409669] Key type ._llcrypt registered [11128.414751] Key type .llcrypt registered [11128.538144] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [11129.474529] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11129.549653] alg: No test for adler32 (adler32-zlib) [11130.577943] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11130.815653] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [11132.464159] Key type lgssc registered [11133.390379] Lustre: Echo OBD driver; http://www.lustre.org/ [11141.328199] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11141.342457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11142.964099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11147.235879] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11152.366023] LustreError: 335187:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11152.381736] LustreError: 335187:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11154.719904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11155.120846] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11158.810290] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11161.217646] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11168.579113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11169.158140] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11176.418931] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11179.436521] LustreError: 336677:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11184.635625] LustreError: 337161:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11185.783394] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11185.993992] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11187.069261] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11187.080678] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11192.372811] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11201.246215] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11205.712030] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11217.901428] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11217.905638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11217.919789] Lustre: Skipped 3 previous similar messages [11217.924613] Lustre: Skipped 2 previous similar messages [11222.918691] Lustre: server umount lustre-MDT0000 complete [11223.019441] LustreError: 335188:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11223.035781] LustreError: 335188:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11226.740752] LustreError: 335170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452218 with bad export cookie 12800730135437967859 [11226.743315] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11226.755530] LustreError: 335170:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11227.269330] Lustre: server umount lustre-MDT0001 complete [11241.639877] Lustre: server umount lustre-OST0000 complete [11244.512201] Lustre: 334451:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788452220/real 1788452220] req@ffff8c6d4aefad80 x1875327972340992/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788452236 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11244.542333] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11244.806401] Lustre: server umount lustre-OST0001 complete [11261.089456] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [11272.337689] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11272.601149] LustreError: 340455:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11272.659389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11276.428379] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11282.918200] LustreError: 340455:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11282.938816] LustreError: 340455:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11285.851437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11286.436745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11290.899490] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11293.653727] Lustre: 341595:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11299.506777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11305.507648] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11312.051386] LustreError: 341949:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11312.416376] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11312.675744] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11312.679680] Lustre: Skipped 1 previous similar message [11318.644073] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11318.789985] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11318.795081] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11325.628123] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11328.703658] Lustre: 343464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11332.580524] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11332.587182] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11332.602652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11332.608812] Lustre: Skipped 1 previous similar message [11334.119069] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11338.088369] Lustre: server umount lustre-MDT0000 complete [11341.648562] LustreError: 340437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452333 with bad export cookie 12800730135437969560 [11341.652769] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11341.659675] LustreError: 340437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11341.924637] Lustre: server umount lustre-MDT0001 complete [11355.394760] Lustre: server umount lustre-OST0000 complete [11368.349695] Lustre: server umount lustre-OST0001 complete [11371.442358] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11372.764637] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11374.609288] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 12:19:23 (1788452363) [11394.405443] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [11401.227936] loop0: detected capacity change from 0 to 1099511627776 [11401.684693] SGI XFS with ACLs, security attributes, quota, no debug enabled [11401.692861] *** VALIDATE xfs *** [11401.705666] XFS (loop0): Mounting V5 Filesystem [11401.766265] XFS (loop0): Ending clean mount [11402.793793] loop1: detected capacity change from 0 to 18691697672192 [11475.709628] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [11483.236279] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [11511.520905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11511.672385] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11511.699293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11511.767887] Lustre: lustre-MDT0000: new disk, initializing [11511.836117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11511.845838] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11514.902278] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11524.679349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11524.934676] Lustre: 348593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11525.016865] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11525.022267] Lustre: Skipped 1 previous similar message [11525.122899] Lustre: lustre-MDT0001: new disk, initializing [11525.351955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11525.363332] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11529.997777] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11540.967722] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11546.487912] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11552.473677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11552.727477] Lustre: lustre-OST0000: new disk, initializing [11552.731895] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11552.737750] Lustre: 350564:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11554.777515] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11554.793344] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11554.901852] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11558.705889] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11566.693719] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11571.834581] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11572.143943] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11577.226548] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11577.440722] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11580.401122] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11580.414465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11580.422594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11580.427167] Lustre: Skipped 4 previous similar messages [11580.438070] Lustre: Skipped 3 previous similar messages [11585.648058] Lustre: server umount lustre-OST0000 complete [11590.627196] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11590.631240] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11590.642430] Lustre: Skipped 1 previous similar message [11590.665711] Lustre: Skipped 2 previous similar messages [11595.447700] Lustre: server umount lustre-MDT0000 complete [11595.747261] LustreError: 348601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11595.768181] LustreError: 348601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11599.150602] LustreError: 349391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452591 with bad export cookie 12800730135437971107 [11599.154775] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11599.161264] LustreError: 349391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11599.547669] Lustre: server umount lustre-MDT0001 complete [11606.189436] XFS (loop0): Unmounting Filesystem [11609.530536] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 12:23:19 (1788452599) [11614.914375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11615.317768] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11615.324560] Lustre: Skipped 2 previous similar messages [11618.712244] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11625.354282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11629.124409] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11635.329899] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11640.147692] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11645.840732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11651.243686] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11657.641412] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11671.013699] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11671.015505] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11671.022600] LustreError: Skipped 1 previous similar message [11671.025251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11671.035042] Lustre: Skipped 1 previous similar message [11671.048569] Lustre: Skipped 1 previous similar message [11676.940937] Lustre: server umount lustre-OST0000 complete [11686.294939] Lustre: server umount lustre-MDT0000 complete [11686.372768] LustreError: 353121:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11686.386697] LustreError: 353121:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [11689.344155] LustreError: 353103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788452681 with bad export cookie 12800730135437971989 [11689.347630] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11689.350419] LustreError: 353103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11689.660642] Lustre: server umount lustre-MDT0001 complete [11699.614684] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [11702.133392] Key type lgssc unregistered [11702.456810] LNet: 356753:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11702.461858] LNetError: 356753:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11702.481602] LNet: Removed LNI 192.168.206.154@tcp [11703.297284] Key type .llcrypt unregistered [11703.303575] Key type ._llcrypt unregistered [11730.018420] Key type ._llcrypt registered [11730.021962] Key type .llcrypt registered [11730.148290] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [11731.201929] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11731.231318] alg: No test for adler32 (adler32-zlib) [11732.273869] Lustre: Lustre: Build Version: 2.17.56_51_g2aa7cea [11732.547476] LNet: Added LNI 192.168.206.154@tcp [8/256/0/180] [11734.241504] Key type lgssc registered [11735.662202] Lustre: Echo OBD driver; http://www.lustre.org/ [11742.943078] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11742.964801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11744.368369] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11749.368386] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11754.989397] LustreError: 358320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11755.002231] LustreError: 358320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11759.185831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11759.891186] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11766.720431] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11774.095317] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11779.233434] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11785.305312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11785.584564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11791.097706] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11797.623700] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11814.600623] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.54@tcp (stopping) [11815.394169] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11815.400429] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11815.408906] Lustre: Skipped 1 previous similar message [11815.431864] Lustre: Skipped 1 previous similar message [11815.715849] Lustre: server umount lustre-MDT0001 complete [11819.691091] LustreError: 360811:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.54@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11824.814188] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.54@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11824.850608] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11828.697274] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11828.816146] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11828.833918] Lustre: Found index 1 for lustre-MDT0001, updating log [11828.902791] Lustre: 358309:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11829.689815] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11834.863204] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11835.282585] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11836.408180] LustreError: 361828:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11836.416935] LustreError: 361828:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11836.433335] LustreError: 358312:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11836.539337] LustreError: 361843:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.154@tcp: cfg command failed: rc = -17 [11836.555510] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11836.555510] [11845.094726] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11845.125260] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11845.127711] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11845.144399] Lustre: Skipped 1 previous similar message [11849.830410] Lustre: server umount lustre-MDT0001 complete [11850.218846] LustreError: 358321:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11850.243864] LustreError: 358321:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11855.334373] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11855.351665] LustreError: 358317:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11862.027684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11862.098790] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11862.102162] Lustre: Found index 1 for lustre-MDT0001, updating log [11862.123874] Lustre: 358308:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11862.343284] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11867.197678] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [11867.625805] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11868.181913] LustreError: 362899:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11868.191654] LustreError: 362899:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11868.212794] LustreError: 358312:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11868.219133] LustreError: 358312:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11868.276253] LustreError: 362913:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11868.281636] LustreError: 362913:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11868.285919] LustreError: 362913:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.154@tcp: cfg command failed: rc = -17 [11868.291948] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.154@tcp [11868.291948] [12172.999622] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.54@tcp (stopping) [12173.007511] Lustre: Skipped 1 previous similar message [12174.819349] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12174.842884] Lustre: Skipped 1 previous similar message [12174.849724] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12178.094071] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.54@tcp (stopping) [12178.112293] Lustre: Skipped 2 previous similar messages [12178.751462] Lustre: server umount lustre-MDT0001 complete [12179.948057] LustreError: 363886:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12179.972039] LustreError: 363886:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12189.377310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12189.664763] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12189.680085] Lustre: Found index 1 for lustre-MDT0001, updating log [12189.835242] Lustre: 358308:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12190.179165] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [12190.196900] Lustre: Skipped 1 previous similar message [12190.624435] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12195.820864] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12196.346439] LustreError: 371797:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12196.363908] LustreError: 371797:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12196.376173] LustreError: 358312:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12196.381627] LustreError: 358312:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12196.443682] LustreError: 371807:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12196.486451] LustreError: 371807:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.206.154@tcp: cfg command failed: rc = -114 [12196.499451] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12196.499451] [12196.718809] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [12518.375333] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12518.378750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12518.386480] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12518.400754] Lustre: Skipped 3 previous similar messages [12518.436570] Lustre: Skipped 2 previous similar messages [12524.802811] Lustre: server umount lustre-MDT0000 complete [12528.610676] LustreError: 358320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12528.628749] LustreError: 358320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [12528.845160] LustreError: 359055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453520 with bad export cookie 15418284079921250878 [12528.853985] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12528.857666] LustreError: 359055:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12533.732694] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12535.333645] Lustre: server umount lustre-MDT0001 complete [12539.171521] Lustre: server umount lustre-OST0000 complete [12548.620314] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [12559.001461] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [12594.749770] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 12:39:43 (1788453583) [12602.719502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12603.355346] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12603.406417] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12603.577767] Lustre: lustre-MDT0000: new disk, initializing [12603.763924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12603.806447] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12610.013991] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [12622.785172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12622.877719] Lustre: 382576:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12622.942137] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12622.946977] Lustre: Skipped 1 previous similar message [12623.038427] Lustre: lustre-MDT0001: new disk, initializing [12623.185300] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12623.222443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12623.236371] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12628.756263] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [12639.833311] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12646.569240] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12654.152804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12654.450167] Lustre: lustre-OST0000: new disk, initializing [12654.453683] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12654.462725] Lustre: 384543:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12654.537985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12661.870924] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12661.887196] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12661.994105] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12662.032195] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_default_debug -1 all [12672.256056] Lustre: DEBUG MARKER: oleg654-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12681.194527] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [12681.203124] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12681.214902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12681.218645] Lustre: Skipped 4 previous similar messages [12682.259378] Lustre: server umount lustre-MDT0000 complete [12682.726917] LustreError: 383882:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12682.754125] LustreError: 383882:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12686.218599] LustreError: 382567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788453678 with bad export cookie 15418284079921252831 [12686.223371] LustreError: MGC192.168.206.154@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12686.230793] LustreError: 382567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12686.650584] Lustre: server umount lustre-MDT0001 complete [12701.397930] Lustre: server umount lustre-OST0000 complete [12732.226171] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing set_hostid [12743.851314] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing load_modules_local [12774.884501] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12519 sec ========= 12:42:44 (1788453764) [12776.649934] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 12:42:45 (1788453765) === [12779.979626] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 12:42:49 (1788453769) === [12818.997479] Lustre: DEBUG MARKER: oleg654-server.virtnet: executing unload_modules_local [12821.808195] Key type lgssc unregistered [12822.132855] LNet: 390914:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12822.138658] LNetError: 390914:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12823.207251] LNet: Removed LNI 192.168.206.154@tcp [12824.110248] Key type .llcrypt unregistered [12824.112171] Key type ._llcrypt unregistered