[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 662356284 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2889096K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.003100] x2apic enabled [ 0.004013] Switched APIC routing to physical x2apic. [ 0.005014] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008057] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009014] pid_max: default: 32768 minimum: 301 [ 0.010186] LSM: Security Framework initializing [ 0.011072] Yama: becoming mindful. [ 0.013025] SELinux: Initializing. [ 0.015072] *** VALIDATE selinux *** [ 0.026179] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.030260] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.031000] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031243] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032159] *** VALIDATE tmpfs *** [ 0.035372] *** VALIDATE proc *** [ 0.037070] *** VALIDATE cgroup *** [ 0.038031] *** VALIDATE cgroup2 *** [ 0.039635] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041180] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043032] Spectre V2 : User space: Vulnerable [ 0.044007] Speculative Store Bypass: Vulnerable [ 0.047601] debug: unmapping init [mem 0xffffffff84459000-0xffffffff84460fff] [ 0.049455] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.051076] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.052023] ... version: 2 [ 0.053013] ... bit width: 48 [ 0.054033] ... generic registers: 4 [ 0.055014] ... value mask: 0000ffffffffffff [ 0.056019] ... max period: 00007fffffffffff [ 0.057016] ... fixed-purpose events: 3 [ 0.058026] ... event mask: 000000070000000f [ 0.059262] rcu: Hierarchical SRCU implementation. [ 0.062092] smp: Bringing up secondary CPUs ... [ 0.063739] x86: Booting SMP configuration: [ 0.064030] .... node #0, CPUs: #1 #2 #3 [ 0.074266] smp: Brought up 1 node, 4 CPUs [ 0.076026] smpboot: Max logical packages: 1 [ 0.077011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.114022] node 0 deferred pages initialised in 35ms [ 0.118149] devtmpfs: initialized [ 0.119536] x86/mm: Memory block size: 128MB [ 0.122845] gcov: version magic: 0x41383552 [ 0.124307] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.125066] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.126364] pinctrl core: initialized pinctrl subsystem [ 0.127188] [ 0.127877] ************************************************************* [ 0.128009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.129010] ** ** [ 0.130009] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.131011] ** ** [ 0.132011] ** This means that this kernel is built to expose internal ** [ 0.133008] ** IOMMU data structures, which may compromise security on ** [ 0.134011] ** your system. ** [ 0.135010] ** ** [ 0.136010] ** If you see this message and you are not debugging the ** [ 0.137024] ** kernel, report this immediately to your vendor! ** [ 0.138015] ** ** [ 0.139012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.140012] ************************************************************* [ 0.141990] NET: Registered protocol family 16 [ 0.142707] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.143056] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.144063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.145766] cpuidle: using governor menu [ 0.148203] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.151789] PCI: Using configuration type 1 for base access [ 0.154202] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.161175] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.163030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.167043] cryptd: max_cpu_qlen set to 1000 [ 0.169471] ACPI: Added _OSI(Module Device) [ 0.170000] ACPI: Added _OSI(Processor Device) [ 0.170000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.172024] ACPI: Added _OSI(Processor Aggregator Device) [ 0.176143] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.184509] ACPI: Interpreter enabled [ 0.186083] ACPI: PM: (supports S0 S3 S4 S5) [ 0.188012] ACPI: Using IOAPIC for interrupt routing [ 0.190151] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.193404] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.205000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.207044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.210020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.214214] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.219000] acpiphp: Slot [2] registered [ 0.219000] acpiphp: Slot [5] registered [ 0.220327] acpiphp: Slot [6] registered [ 0.223357] acpiphp: Slot [7] registered [ 0.225245] acpiphp: Slot [8] registered [ 0.226164] acpiphp: Slot [9] registered [ 0.228137] acpiphp: Slot [10] registered [ 0.230106] acpiphp: Slot [3] registered [ 0.232101] acpiphp: Slot [4] registered [ 0.233129] acpiphp: Slot [11] registered [ 0.235085] acpiphp: Slot [12] registered [ 0.236390] acpiphp: Slot [13] registered [ 0.238206] acpiphp: Slot [14] registered [ 0.240075] acpiphp: Slot [15] registered [ 0.242097] acpiphp: Slot [16] registered [ 0.244122] acpiphp: Slot [17] registered [ 0.246094] acpiphp: Slot [18] registered [ 0.247171] acpiphp: Slot [19] registered [ 0.249075] acpiphp: Slot [20] registered [ 0.250238] acpiphp: Slot [21] registered [ 0.253119] acpiphp: Slot [22] registered [ 0.254081] acpiphp: Slot [23] registered [ 0.256131] acpiphp: Slot [24] registered [ 0.258103] acpiphp: Slot [25] registered [ 0.259185] acpiphp: Slot [26] registered [ 0.261176] acpiphp: Slot [27] registered [ 0.263106] acpiphp: Slot [28] registered [ 0.264100] acpiphp: Slot [29] registered [ 0.266156] acpiphp: Slot [30] registered [ 0.267339] acpiphp: Slot [31] registered [ 0.269416] PCI host bridge to bus 0000:00 [ 0.272064] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.274019] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.277022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.279136] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.283026] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.289049] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.291163] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.295363] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.298740] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.312463] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.318051] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.320026] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.323025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.326019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.329311] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.332924] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.337037] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.341038] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.347020] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.363019] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.370011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.377687] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.391015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.401017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.418018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.437854] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.453015] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.467022] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.507020] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.527000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.547031] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.562021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.598034] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.624743] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.639016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.656028] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.685031] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.709710] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.729021] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.741022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.788030] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.808317] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.825019] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.834028] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.872018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.893000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.898883] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.901347] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.904588] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.908265] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.917073] iommu: Default domain type: Passthrough [ 0.925000] SCSI subsystem initialized [ 0.930236] ACPI: bus type USB registered [ 0.932194] usbcore: registered new interface driver usbfs [ 0.937114] usbcore: registered new interface driver hub [ 0.942117] usbcore: registered new device driver usb [ 0.945174] pps_core: LinuxPPS API ver. 1 registered [ 0.947017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.954122] PTP clock support registered [ 0.956074] EDAC MC: Ver: 3.0.0 [ 0.958000] PCI: Using ACPI for IRQ routing [ 0.958000] NetLabel: Initializing [ 0.960020] NetLabel: domain hash size = 128 [ 0.963011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.968086] NetLabel: unlabeled traffic allowed by default [ 0.970073] vgaarb: loaded [ 0.978307] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.980013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.991794] clocksource: Switched to clocksource kvm-clock [ 1.288157] VFS: Disk quotas dquot_6.6.0 [ 1.289615] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.295497] *** VALIDATE ramfs *** [ 1.298979] *** VALIDATE hugetlbfs *** [ 1.304061] pnp: PnP ACPI init [ 1.308527] pnp: PnP ACPI: found 6 devices [ 1.361883] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.368924] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.372402] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.375985] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.379736] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.384011] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.388752] NET: Registered protocol family 2 [ 1.393388] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.455732] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.463156] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.472024] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.477811] TCP: Hash tables configured (established 65536 bind 65536) [ 1.481104] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.487076] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.492510] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.498129] NET: Registered protocol family 1 [ 1.501962] RPC: Registered named UNIX socket transport module. [ 1.504733] RPC: Registered udp transport module. [ 1.507215] RPC: Registered tcp transport module. [ 1.509680] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.512658] NET: Registered protocol family 44 [ 1.514407] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.549844] pci 0000:00:00.0: quirk_natoma+0x0/0x20 took 34606 usecs [ 1.552457] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.555026] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.557364] PCI: CLS 0 bytes, default 64 [ 1.558817] Unpacking initramfs... [ 4.319387] debug: unmapping init [mem 0xffff8c343cc54000-0xffff8c343ffbffff] [ 4.330731] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 4.336509] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 4.339349] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 5.098832] Initialise system trusted keyrings [ 5.101292] Key type blacklist registered [ 5.105985] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.127788] zbud: loaded [ 5.132907] *** VALIDATE nfs *** [ 5.133801] *** VALIDATE nfs4 *** [ 5.137085] pstore: using deflate compression [ 5.148440] Platform Keyring initialized [ 5.393686] NET: Registered protocol family 38 [ 5.395700] Key type asymmetric registered [ 5.397734] Asymmetric key parser 'x509' registered [ 5.399658] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 5.402890] io scheduler mq-deadline registered [ 5.404334] io scheduler kyber registered [ 5.405744] io scheduler bfq registered [ 5.407682] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 5.409920] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 5.412487] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 5.421458] ACPI: Power Button [PWRF] [ 5.435384] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 5.445838] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 5.466938] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 5.480495] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 5.543180] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 5.585254] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.639663] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.646654] Non-volatile memory driver v1.3 [ 5.647821] Linux agpgart interface v0.103 [ 5.694509] virtio_blk virtio1: [vda] 134136 512-byte logical blocks (68.7 MB/65.5 MiB) [ 5.701088] vda: detected capacity change from 0 to 68677632 [ 5.753912] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.759950] vdb: detected capacity change from 0 to 1073741824 [ 5.811832] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.815762] vdc: detected capacity change from 0 to 2621440000 [ 5.856066] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.863041] vdd: detected capacity change from 0 to 2621440000 [ 5.900577] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.903466] vde: detected capacity change from 0 to 4294967296 [ 5.962782] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.978506] vdf: detected capacity change from 0 to 4294967296 [ 6.005796] libphy: Fixed MDIO Bus: probed [ 6.031902] usbcore: registered new interface driver usbserial_generic [ 6.043450] usbserial: USB Serial support registered for generic [ 6.050231] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.067247] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.072179] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.080190] mousedev: PS/2 mouse device common for all mice [ 6.087400] rtc_cmos 00:05: RTC can wake from S4 [ 6.090779] rtc_cmos 00:05: registered as rtc0 [ 6.093401] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.095778] intel_pstate: CPU model not supported [ 6.097470] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.103860] hid: raw HID events driver (C) Jiri Kosina [ 6.106499] usbcore: registered new interface driver usbhid [ 6.109458] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.110369] usbhid: USB HID core driver [ 6.110491] drop_monitor: Initializing network drop monitor service [ 6.110682] Initializing XFRM netlink socket [ 6.111044] NET: Registered protocol family 10 [ 6.117790] Segment Routing with IPv6 [ 6.122553] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.123862] NET: Registered protocol family 17 [ 6.124255] mpls_gso: MPLS GSO support [ 6.147952] RAS: Correctable Errors collector initialized. [ 6.153707] AVX version of gcm_enc/dec engaged. [ 6.157452] AES CTR mode by8 optimization enabled [ 6.489154] sched_clock: Marking stable (6489113988, 0)->(8082361978, -1593247990) [ 6.541790] registered taskstats version 1 [ 6.549199] Loading compiled-in X.509 certificates [ 6.558435] zswap: loaded using pool lzo/zbud [ 6.724748] Key type big_key registered [ 6.758499] Key type encrypted registered [ 6.759862] ima: No TPM chip found, activating TPM-bypass! [ 6.761575] ima: Allocated hash algorithm: sha1 [ 6.768477] ima: No architecture policies found [ 6.774366] evm: Initialising EVM extended attributes: [ 6.776972] evm: security.selinux [ 6.779406] evm: security.ima [ 6.782359] evm: security.capability [ 6.785906] evm: HMAC attrs: 0x1 [ 6.798474] rtc_cmos 00:05: setting system clock to 2026-04-10 18:42:49 UTC (1775846569) [ 6.815234] debug: unmapping init [mem 0xffffffff85403000-0xffffffff855fffff] [ 6.836267] debug: unmapping init [mem 0xffffffff84182000-0xffffffff84458fff] [ 6.848242] Write protecting the kernel read-only data: 28672k [ 6.860222] debug: unmapping init [mem 0xffffffff82803000-0xffffffff829fffff] [ 6.867597] debug: unmapping init [mem 0xffffffff83114000-0xffffffff831fffff] [ 6.992706] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.016499] systemd[1]: Detected virtualization kvm. [ 7.023937] systemd[1]: Detected architecture x86-64. [ 7.029763] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.232438] systemd[1]: No hostname configured. [ 7.239694] systemd[1]: Set hostname to . [ 7.249868] random: systemd: uninitialized urandom read (16 bytes read) [ 7.265175] systemd[1]: Initializing machine ID from random generator. [ 7.608280] random: systemd: uninitialized urandom read (16 bytes read) [ 7.613203] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 7.620689] random: systemd: uninitialized urandom read (16 bytes read) [ 7.623245] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 7.637387] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... Starting Journal Service... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Kernel Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 9.969445] device-mapper: uevent: version 1.0.3 [ 9.975099] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 11.472487] virtio_net virtio0 ens2: renamed from eth0 [ 11.511816] random: fast init done [ 12.259049] scsi host0: ata_piix [ 12.611725] scsi host1: ata_piix [ 12.612968] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.630667] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 16.323426] random: crng init done [ 16.329408] random: 7 urandom warning(s) missed due to ratelimiting [ 20.073350] dracut-initqueue[593]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 22.324087] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 26.223463] printk: systemd: 26 output lines suppressed due to ratelimiting [ 26.994762] SELinux: Disabled at runtime. [ 27.124894] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 27.140168] systemd[1]: Detected virtualization kvm. [ 27.143197] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 28.030473] systemd[1]: initrd-switch-root.service: Succeeded. [ 28.035605] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 28.050518] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 28.058546] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 28.064731] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 28.080381] systemd[1]: Starting Journal Service... Starting Journal Service... [ 28.116017] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Listening on udev Control Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Started Forward Password Requests to Wall Directory Watch. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... Mounting Kernel Debug File System... Starting Create list of required st…ce nodes for the current kernel... [ 28.488930] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting udev Coldplug all Devices... Starting Remount Root and Kernel File Systems... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ 29.568102] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 30.588962] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 30.818647] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 31.294229] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 31.340450] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (7s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (8s / no limit)[ 37.203064] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 38.267848] NFS: Registering the id_resolver key type [ 38.270643] Key type id_resolver registered [ 38.274338] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. Starting Login Service... Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started Login Service. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg227-server login: [ 84.311541] libcfs: loading out-of-tree module taints kernel. [ 84.341862] Key type ._llcrypt registered [ 84.347691] Key type .llcrypt registered [ 84.419942] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 95.411798] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 96.539256] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 96.551847] alg: No test for adler32 (adler32-zlib) [ 97.731737] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 98.263645] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 99.935273] Key type lgssc registered [ 101.047818] Lustre: Echo OBD driver; http://www.lustre.org/ [ 110.181808] hrtimer: interrupt took 10497637 ns [ 113.033963] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 141.105738] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 150.508390] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 150.553663] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 151.753912] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 151.778852] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 151.866528] Lustre: lustre-MDT0000: new disk, initializing [ 151.926346] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 151.935336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 154.783618] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 165.291306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.362191] Lustre: 6489:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 165.411947] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 165.413808] Lustre: Skipped 1 previous similar message [ 165.501931] Lustre: lustre-MDT0001: new disk, initializing [ 165.551884] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 165.583687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 165.601149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 168.252450] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.794605] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 177.818159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 178.060899] Lustre: lustre-OST0000: new disk, initializing [ 178.070944] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 178.125276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 182.211811] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 183.300899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 183.307176] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 183.334646] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000400 [ 191.328933] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 191.434233] Lustre: lustre-OST0001: new disk, initializing [ 191.439752] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 191.493284] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 195.817099] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.128806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 196.137754] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 196.186695] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 205.690733] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 209.881377] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 218.193108] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing check_logdir /tmp/testlogs/ [ 221.161790] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing yml_node [ 223.758431] Lustre: DEBUG MARKER: Client: 2.17.52.1 [ 225.603403] Lustre: DEBUG MARKER: MDS: 2.17.52.1 [ 227.746068] Lustre: DEBUG MARKER: OSS: 2.17.52.1 [ 228.818312] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Apr 10 14:46:30 EDT 2026 [ 241.108606] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 242.107656] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 257.506245] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 257.506911] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 257.523018] Lustre: Skipped 2 previous similar messages [ 257.527512] Lustre: Skipped 3 previous similar messages [ 261.512106] Lustre: server umount lustre-MDT0000 complete [ 263.845302] LustreError: 6480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775846826 with bad export cookie 2453482512457360180 [ 263.846975] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 263.850416] LustreError: 6480:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 267.743791] LustreError: 6494:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 267.744965] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 267.745828] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 267.761576] LustreError: 6494:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 267.779824] Lustre: Skipped 2 previous similar messages [ 268.768391] LustreError: 6496:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 268.772685] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 268.776540] LustreError: 6496:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 268.780301] Lustre: Skipped 2 previous similar messages [ 270.062745] Lustre: server umount lustre-MDT0001 complete [ 272.362738] Lustre: server umount lustre-OST0000 complete [ 274.310295] Lustre: server umount lustre-OST0001 complete [ 277.493410] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 282.060290] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 302.379732] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 302.548443] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 302.576557] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 302.663052] Lustre: lustre-MDT0000: new disk, initializing [ 302.740822] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 302.755080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 305.080388] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 312.225081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 312.307381] Lustre: 14953:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 312.353165] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 312.356098] Lustre: Skipped 1 previous similar message [ 312.399663] Lustre: lustre-MDT0001: new disk, initializing [ 312.452865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 312.470530] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 312.477831] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 315.001465] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 319.980717] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 321.899421] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 326.478265] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 328.542431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 328.552332] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 328.617098] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 329.828409] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 334.418305] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 336.505744] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 336.638605] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 339.081928] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 339.223623] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 344.037332] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 344.039853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 344.046382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 344.048295] Lustre: Skipped 1 previous similar message [ 344.059284] LustreError: Skipped 1 previous similar message [ 346.608459] Lustre: server umount lustre-OST0000 complete [ 349.153926] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 349.163133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 349.165760] Lustre: Skipped 2 previous similar messages [ 355.027720] Lustre: server umount lustre-MDT0000 complete [ 357.276451] LustreError: 14945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775846919 with bad export cookie 2453482512457361720 [ 357.279428] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 357.281772] LustreError: 14945:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 357.437565] Lustre: server umount lustre-MDT0001 complete [ 360.358701] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 14:48:42 (1775846922) [ 364.745238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 365.011648] LustreError: 18812:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 365.053803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 365.060901] Lustre: Skipped 1 previous similar message [ 367.253919] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 370.144489] LustreError: 18813:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 372.002241] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 374.155602] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 377.023889] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 378.964659] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 382.923872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 386.866303] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 389.820808] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 401.533826] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 407.012968] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 407.017907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 407.027739] Lustre: Skipped 2 previous similar messages [ 407.033565] Lustre: Skipped 2 previous similar messages [ 410.965382] Lustre: server umount lustre-MDT0000 complete [ 412.129268] LustreError: 20423:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 412.146129] LustreError: 20423:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 413.005563] LustreError: 18793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775846975 with bad export cookie 2453482512457362602 [ 413.011145] LustreError: 18793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 413.011946] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 413.202488] Lustre: server umount lustre-MDT0001 complete [ 425.264590] Lustre: server umount lustre-OST0000 complete [ 436.775350] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 442.439362] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 442.724528] LustreError: 22855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 442.772393] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 442.774967] Lustre: Skipped 2 previous similar messages [ 444.672969] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 449.171818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 451.733269] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 453.383638] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 457.390355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 461.386617] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 466.729039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 466.843630] Lustre: lustre-OST0001: new disk, initializing [ 466.847131] Lustre: Skipped 1 previous similar message [ 466.850065] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 466.859627] Lustre: Skipped 1 previous similar message [ 468.436440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 468.443069] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 468.513398] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 470.521306] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 477.221341] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 479.447378] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 493.895979] Lustre: server umount lustre-MDT0000 complete [ 494.049354] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 494.059082] Lustre: Skipped 2 previous similar messages [ 494.070262] LustreError: 22852:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 494.095660] LustreError: 22852:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 496.343631] LustreError: 22835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847059 with bad export cookie 2453482512457363449 [ 496.344870] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 496.356937] LustreError: 22835:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 496.578224] Lustre: server umount lustre-MDT0001 complete [ 508.379188] Lustre: server umount lustre-OST0000 complete [ 521.042767] Lustre: server umount lustre-OST0001 complete [ 529.531943] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 535.683631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 535.969234] LustreError: 28097:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 536.038162] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 536.041015] Lustre: Skipped 3 previous similar messages [ 538.604084] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 543.971493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 547.050612] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 548.814491] Lustre: 29209:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 552.852868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 556.246905] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 561.550384] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 565.834143] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 570.704154] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 572.679352] Lustre: 31061:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 582.113664] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 582.122832] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 582.129690] Lustre: Skipped 5 previous similar messages [ 582.134055] Lustre: Skipped 4 previous similar messages [ 586.428693] Lustre: server umount lustre-MDT0000 complete [ 588.179902] LustreError: 28078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847150 with bad export cookie 2453482512457364744 [ 588.182345] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 588.185951] LustreError: 28078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 588.337724] Lustre: server umount lustre-MDT0001 complete [ 600.602977] Lustre: server umount lustre-OST0000 complete [ 612.790988] Lustre: server umount lustre-OST0001 complete [ 617.559940] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:52:59 (1775847179) [ 632.407982] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 637.890612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 638.131874] LustreError: 33841:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 638.137619] LustreError: 33841:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 10 previous similar messages [ 640.599523] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 645.801071] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 648.158555] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 649.739284] Lustre: 34954:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 653.543043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 656.993853] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 661.842026] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 665.377212] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 669.558315] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 671.414135] Lustre: 36806:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 681.958832] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 681.965218] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 681.971726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 681.974340] Lustre: Skipped 1 previous similar message [ 684.940928] Lustre: server umount lustre-MDT0000 complete [ 686.836331] LustreError: 33823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847249 with bad export cookie 2453482512457365843 [ 686.837475] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 686.847597] LustreError: 33823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 687.040855] Lustre: server umount lustre-MDT0001 complete [ 698.610128] Lustre: server umount lustre-OST0000 complete [ 711.094599] Lustre: server umount lustre-OST0001 complete [ 714.990781] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:54:36 (1775847276) [ 722.872228] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 728.820118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 729.128785] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 729.131644] Lustre: Skipped 7 previous similar messages [ 731.460795] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 736.079951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 738.733572] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 740.286724] Lustre: 40043:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 743.745639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 746.594578] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 750.563558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 753.123254] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 757.312837] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 764.604825] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 764.608793] Lustre: Skipped 1 previous similar message [ 766.945255] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 766.954704] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 766.961120] Lustre: Skipped 3 previous similar messages [ 766.963747] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 766.969066] Lustre: Skipped 4 previous similar messages [ 772.495518] Lustre: server umount lustre-MDT0000 complete [ 774.116862] LustreError: 38927:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 774.131761] LustreError: 38927:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 774.636829] LustreError: 38912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847337 with bad export cookie 2453482512457366956 [ 774.641252] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 774.642975] LustreError: 38912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 780.966730] Lustre: server umount lustre-MDT0001 complete [ 786.462182] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 799.716896] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x220c855891a78252 [ 799.733392] Lustre: MGC192.168.202.127@tcp: Connection restored to 0@lo (at 0@lo) [ 802.323914] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 807.084931] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 809.396104] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 809.485177] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 812.481807] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 813.353805] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 815.283844] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 816.270680] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 819.562484] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 819.565579] Lustre: Skipped 1 previous similar message [ 824.722783] Lustre: server umount lustre-MDT0000 complete [ 826.896616] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847389 with bad export cookie 2453482512457368146 [ 826.897826] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 826.912039] LustreError: 38911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 827.074034] Lustre: server umount lustre-MDT0001 complete [ 839.199908] Lustre: server umount lustre-OST0000 complete [ 851.389513] Lustre: server umount lustre-OST0001 complete [ 856.004678] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:56:57 (1775847417) [ 865.336355] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 870.499612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 872.907366] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 877.600265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 879.972895] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 881.520042] Lustre: 47381:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 881.525436] Lustre: 47381:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 884.934557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 887.734804] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 891.612530] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 894.731585] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 898.950856] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 906.525534] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 913.888984] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 913.896224] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 913.909644] Lustre: Skipped 9 previous similar messages [ 913.914046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 913.918083] Lustre: Skipped 13 previous similar messages [ 919.322264] Lustre: server umount lustre-MDT0000 complete [ 921.261345] LustreError: 47383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847483 with bad export cookie 2453482512457368944 [ 921.263520] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 921.267436] LustreError: 47383:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 921.530986] Lustre: server umount lustre-MDT0001 complete [ 933.431369] Lustre: server umount lustre-OST0000 complete [ 945.739893] Lustre: server umount lustre-OST0001 complete [ 951.371276] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:58:33 (1775847513) [ 952.738230] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 953.786934] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:58:35 (1775847515) [ 959.128662] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 962.947353] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 974.349089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 974.512696] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 974.554936] Lustre: lustre-MDT0000: new disk, initializing [ 974.611636] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 976.535830] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 980.569782] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 983.922163] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 985.325031] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 985.341521] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 986.869322] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 991.298757] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 992.523925] Lustre: DEBUG MARKER: create test files [ 1011.171301] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1013.091495] Lustre: server umount lustre-OST0000 complete [ 1015.276642] Lustre: server umount lustre-MDT0000 complete [ 1020.532216] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 1022.080129] Key type lgssc unregistered [ 1022.277034] LNet: 54778:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1022.281415] LNetError: 54778:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1022.293477] LNet: Removed LNI 192.168.202.127@tcp [ 1022.824116] Key type .llcrypt unregistered [ 1022.825383] Key type ._llcrypt unregistered [ 1036.520694] Key type ._llcrypt registered [ 1036.522446] Key type .llcrypt registered [ 1036.573476] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1037.035390] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1037.096779] alg: No test for adler32 (adler32-zlib) [ 1038.006147] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 1038.143895] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 1039.735243] Key type lgssc registered [ 1040.184763] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1043.798582] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1043.808905] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1045.023985] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1046.763343] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1049.189548] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1052.428178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1052.611871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1054.709112] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1054.751215] Lustre: 57359:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1054.757371] Lustre: 57359:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1054.760231] Lustre: 57359:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1054.763345] Lustre: 57359:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/24/1, punch: 0/0/0, quota 1/3/0 [ 1054.765991] Lustre: 57359:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1054.768232] Lustre: 57359:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1055.269739] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1057.863285] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1061.280420] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1065.110703] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1070.051072] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1070.051590] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1070.076950] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1075.174455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1075.821607] Lustre: server umount lustre-OST0000 complete [ 1078.778713] Lustre: server umount lustre-MDT0000 complete [ 1086.937602] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 1088.637222] Key type lgssc unregistered [ 1088.829827] LNet: 58837:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1088.834767] LNetError: 58837:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1088.843528] LNet: Removed LNI 192.168.202.127@tcp [ 1089.309731] Key type .llcrypt unregistered [ 1089.311979] Key type ._llcrypt unregistered [ 1108.946169] Key type ._llcrypt registered [ 1108.948196] Key type .llcrypt registered [ 1109.020950] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1109.703981] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1109.722348] alg: No test for adler32 (adler32-zlib) [ 1110.675780] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 1110.831156] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 1112.455180] Key type lgssc registered [ 1113.033281] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1117.849546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1117.862393] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1119.140620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1121.721924] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1124.945407] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1129.147969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1129.361038] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1134.265698] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1144.077291] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1144.150825] Lustre: 61531:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1144.157187] Lustre: 61531:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1144.162036] Lustre: 61531:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1144.166590] Lustre: 61531:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/24/1, punch: 0/0/0, quota 1/3/0 [ 1144.174562] Lustre: 61531:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1144.178492] Lustre: 61531:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1146.611663] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1151.679078] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1164.256397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1164.288918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1168.909501] Lustre: server umount lustre-OST0000 complete [ 1174.880462] Lustre: server umount lustre-MDT0000 complete [ 1187.684331] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 1190.503832] Key type lgssc unregistered [ 1190.796951] LNet: 62963:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1190.803397] LNetError: 62963:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1191.845379] LNet: Removed LNI 192.168.202.127@tcp [ 1192.869043] Key type .llcrypt unregistered [ 1192.871946] Key type ._llcrypt unregistered [ 1210.427297] Key type ._llcrypt registered [ 1210.430224] Key type .llcrypt registered [ 1210.510228] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 1226.624540] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1227.886426] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1228.171025] alg: No test for adler32 (adler32-zlib) [ 1229.538890] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 1229.980534] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 1231.751147] Key type lgssc registered [ 1233.212225] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1270.001516] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1270.028298] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1271.278473] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1271.311765] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1271.399491] Lustre: lustre-MDT0000: new disk, initializing [ 1271.482383] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1271.510453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1276.454959] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1286.045635] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1286.145393] Lustre: 66081:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1286.225695] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1286.230791] Lustre: Skipped 1 previous similar message [ 1286.324499] Lustre: lustre-MDT0001: new disk, initializing [ 1286.430670] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1286.474925] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1286.500616] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1290.782735] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1298.307769] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1301.684414] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1308.198288] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1308.499283] Lustre: lustre-OST0000: new disk, initializing [ 1308.501905] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1308.609422] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1314.233705] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1314.868145] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1314.883647] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1315.001554] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1320.694260] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1324.718337] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1324.914911] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1328.929299] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1329.149252] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1330.660136] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1330.664402] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1330.670117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1330.679645] Lustre: Skipped 1 previous similar message [ 1335.787363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1335.797444] Lustre: Skipped 1 previous similar message [ 1336.955753] Lustre: server umount lustre-OST0000 complete [ 1340.898981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1340.910120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1340.915204] Lustre: Skipped 2 previous similar messages [ 1346.017675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1346.022925] Lustre: Skipped 2 previous similar messages [ 1346.954586] Lustre: server umount lustre-MDT0000 complete [ 1350.158586] LustreError: 66073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775847912 with bad export cookie 4771143048540440212 [ 1350.173444] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1350.412415] Lustre: server umount lustre-MDT0001 complete [ 1358.216532] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:05:19 (1775847919) [ 1368.180380] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1405.962287] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 1417.036856] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1452.835941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1453.008682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1453.045406] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1453.093284] Lustre: lustre-MDT0000: new disk, initializing [ 1453.162406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1453.177949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1457.021557] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1466.868888] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1467.029729] Lustre: 73640:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1467.093765] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1467.097708] Lustre: Skipped 1 previous similar message [ 1467.165290] Lustre: lustre-MDT0001: new disk, initializing [ 1467.280249] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1467.326561] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1467.340438] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1472.113710] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1479.510145] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1483.046404] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1488.951408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1489.147455] Lustre: lustre-OST0000: new disk, initializing [ 1489.152574] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1489.223394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1494.590476] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1496.097218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1496.104360] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1496.178373] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1500.738410] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1505.008435] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1505.210105] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1509.202867] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1509.380643] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1511.399335] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1511.406441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1511.407598] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1511.412397] Lustre: Skipped 1 previous similar message [ 1511.429228] LustreError: Skipped 1 previous similar message [ 1517.323624] Lustre: server umount lustre-OST0000 complete [ 1521.634944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1521.638068] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1521.639909] Lustre: Skipped 2 previous similar messages [ 1521.642835] Lustre: Skipped 2 previous similar messages [ 1526.265299] Lustre: server umount lustre-MDT0000 complete [ 1528.996923] LustreError: 73635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775848091 with bad export cookie 4771143048540441094 [ 1529.003048] LustreError: 73635:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1529.003194] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1529.299200] Lustre: server umount lustre-MDT0001 complete [ 1535.343614] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:08:16 (1775848096) [ 1539.976655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1540.266667] LustreError: 77644:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1540.277792] LustreError: 77644:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1540.317859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1543.208875] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1545.700531] LustreError: 77645:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1549.795537] LustreError: 77644:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1549.930101] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1554.393975] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1558.653201] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1561.455860] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1566.941330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1571.948721] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1576.566300] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1579.660463] Lustre: 79982:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting 1ddc5750-cbfe-4ac8-9e17-3fa3ace8a322 at adminstrative request [ 1584.545702] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1584.571326] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1594.869972] Lustre: MGS: Client 1ddc5750-cbfe-4ac8-9e17-3fa3ace8a322 (at 0@lo) reconnecting [ 1594.893471] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x42368152f4d3cd78 to 0x42368152f4d3d018 [ 1594.930754] Lustre: MGC192.168.202.127@tcp: Connection restored to 0@lo (at 0@lo) [ 1622.794490] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1622.987156] Lustre: lustre-OST0001: new disk, initializing [ 1622.991150] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1623.114298] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1623.117109] Lustre: Skipped 2 previous similar messages [ 1629.810191] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1632.811404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1632.818241] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1632.929487] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1636.850708] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1642.979334] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1642.995156] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1643.016608] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1643.022430] Lustre: Skipped 1 previous similar message [ 1645.686435] Lustre: server umount lustre-OST0001 complete [ 1648.621873] LustreError: 80022:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1648.640749] LustreError: 80022:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1651.680050] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1651.682703] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1651.686118] LustreError: Skipped 2 previous similar messages [ 1651.691673] Lustre: Skipped 1 previous similar message [ 1653.731519] LustreError: 79239:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1653.743794] LustreError: 79239:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1655.244673] Lustre: server umount lustre-OST0000 complete [ 1662.946131] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1665.475807] Lustre: server umount lustre-MDT0000 complete [ 1669.089807] LustreError: 77644:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1669.096397] LustreError: 77644:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1669.101711] LustreError: 77625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775848231 with bad export cookie 4771143048540442648 [ 1669.105916] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1669.106169] LustreError: 77625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1669.328280] Lustre: server umount lustre-MDT0001 complete [ 1676.666291] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:10:37 (1775848237) [ 1678.059594] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1679.622602] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:10:40 (1775848240) [ 1681.215749] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1682.698695] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:10:43 (1775848243) [ 1684.419798] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1686.069546] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 15:10:47 (1775848247) [ 1687.503167] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1689.153595] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:10:50 (1775848250) [ 1690.911778] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1696.669992] loop: module loaded [ 1696.687501] loop0: detected capacity change from 0 to 4294967296 [ 1696.725803] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1698.011439] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1700.987424] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1704.018185] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1723.565177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1723.606293] LustreError: 84376:0:(obd_config.c:843:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1723.613608] LustreError: 84376:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1723.616315] LustreError: 84376:0:(tgt_mount.c:2513:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1723.619566] LustreError: 84376:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1736.952574] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 1743.532379] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 1766.234725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1766.356435] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1766.374675] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1766.442329] Lustre: lustre-MDT0000: new disk, initializing [ 1766.509658] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1766.523443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1769.146826] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1775.466761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1775.547473] Lustre: 87203:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1775.729945] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1778.254911] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1783.454351] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1786.772678] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1796.807595] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1802.910459] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1803.140769] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1809.487846] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1820.655748] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1828.909189] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1829.456565] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1837.958234] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1838.556837] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1844.204753] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1844.219451] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1844.241194] Lustre: Skipped 3 previous similar messages [ 1844.272589] Lustre: Skipped 10 previous similar messages [ 1848.121716] Lustre: server umount lustre-OST0000 complete [ 1857.515386] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1862.891706] Lustre: server umount lustre-MDT0000 complete [ 1864.679524] LustreError: 89606:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1864.711614] LustreError: 89606:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1868.445125] LustreError: 88943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775848431 with bad export cookie 4771143048540443264 [ 1868.448561] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1868.459048] LustreError: 88943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1869.136487] Lustre: server umount lustre-MDT0001 complete [ 1880.249328] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:14:00 (1775848440) [ 1887.681705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1892.904207] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1897.441529] LustreError: 91212:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1897.456499] LustreError: 91212:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 1902.185483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1902.458208] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 1902.460744] Lustre: Skipped 3 previous similar messages [ 1907.263279] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1913.472822] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1917.832276] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1924.710151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1930.832349] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1936.905231] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1941.508408] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1941.715958] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1947.141845] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1947.381887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1954.784054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1955.071373] Lustre: lustre-OST0001: new disk, initializing [ 1955.076538] Lustre: Skipped 2 previous similar messages [ 1955.088800] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1955.092956] Lustre: Skipped 3 previous similar messages [ 1961.437300] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 1962.605883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1962.615573] Lustre: Skipped 2 previous similar messages [ 1962.624844] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1962.722568] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1969.888287] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1975.166822] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1975.456633] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1982.879528] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1983.256760] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2006.392896] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2035.332984] Lustre: Failing over lustre-MDT0000 [ 2035.682568] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2035.690530] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.718183] Lustre: Skipped 2 previous similar messages [ 2035.729864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2035.733081] Lustre: Skipped 3 previous similar messages [ 2036.153395] Lustre: server umount lustre-MDT0000 complete [ 2037.961306] LustreError: 91207:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.27@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2037.988458] LustreError: 91207:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2056.160535] Lustre: 64306:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775848602/real 1775848602] req@000000002f45c017 x1862111374613632/t0(0) o400->MGC192.168.202.127@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1775848618 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2056.204976] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2062.639889] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2062.643119] LDISKFS-fs (dm-0): recovery complete [ 2062.652651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2065.440055] LustreError: 64302:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000005886d899 x1862111374621312/t0(0) o250->MGC192.168.202.127@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2065.888159] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2069.664801] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2069.674128] Lustre: 95973:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2070.923902] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2071.012675] Lustre: 91925:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2071.025154] Lustre: 91925:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 2071.047171] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2071.143698] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2071.495589] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2071.839350] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2071.848155] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2072.168500] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2072.181118] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 2072.198542] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2072.222774] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 7 previous similar messages [ 2072.879981] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2072.887362] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2073.215604] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2073.231117] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 2 previous similar messages [ 2074.501089] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2074.512765] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 2075.148780] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2075.161480] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2075.487480] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2075.498030] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 2078.641663] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2078.654719] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 2079.298723] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2079.316856] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2079.648945] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2079.665859] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 2086.730759] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2086.747327] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 24 previous similar messages [ 2087.411623] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2087.421098] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 2087.743164] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2087.752070] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 24 previous similar messages [ 2103.041169] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2103.073519] Lustre: 96921:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 50 previous similar messages [ 2103.722982] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2103.758821] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2104.096516] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2104.108778] LustreError: 96921:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 50 previous similar messages [ 2129.063151] Lustre: 96921:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2129.075608] Lustre: 96921:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2129.095285] Lustre: 96921:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client cc041435-75f5-4e42-be2e-c28069570706@192.168.202.27@tcp [ 2129.122354] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2129.136290] Lustre: 96921:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@0000000040ec0e1b x1862111356589952/t0(8589934777) o101->cc041435-75f5-4e42-be2e-c28069570706@192.168.202.27@tcp:367/0 lens 592/0 e 0 to 0 dl 1775848702 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2129.194975] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2129.206497] Lustre: lustre-MDT0000: Denying connection for new client cc041435-75f5-4e42-be2e-c28069570706 (at 192.168.202.27@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2129.245666] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2129.333411] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2129.335859] Lustre: Skipped 2 previous similar messages [ 2129.349872] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2129.442153] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2129.442805] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2152.942136] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2152.953431] Lustre: Skipped 4 previous similar messages [ 2156.741996] Lustre: server umount lustre-OST0000 complete [ 2162.660068] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2167.095818] Lustre: server umount lustre-OST0001 complete [ 2173.424707] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2173.431574] Lustre: Skipped 6 previous similar messages [ 2177.963651] Lustre: server umount lustre-MDT0000 complete [ 2178.548755] LustreError: 91925:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2178.577081] LustreError: 91925:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 45 previous similar messages [ 2181.860115] LustreError: 91194:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775848744 with bad export cookie 4771143048540494483 [ 2181.871183] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2181.874841] LustreError: 91194:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2182.176623] Lustre: server umount lustre-MDT0001 complete [ 2191.379588] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 15:19:12 (1775848752) [ 2199.186060] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2199.193986] Lustre: *** cfs_fail_loc=197, val=0*** [ 2214.752396] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2219.871497] LustreError: 98882:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.127@tcp: failed processing log, type 4: rc = -110 [ 2225.024943] Lustre: *** cfs_fail_loc=197, val=0*** [ 2225.033311] Lustre: Skipped 65 previous similar messages [ 2245.600863] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2245.614675] Lustre: Skipped 3 previous similar messages [ 2254.325932] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2260.974753] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2264.023517] Lustre: server umount lustre-OST0000 complete [ 2277.765840] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 15:20:37 (1775848837) [ 2300.575377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2305.663793] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2313.381373] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2317.680305] LustreError: 100425:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2317.695930] LustreError: 100425:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2317.701862] LustreError: 100425:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 2317.994623] Lustre: server umount lustre-MDT0000 complete [ 2352.546344] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 2361.935680] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 2404.712540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2405.212694] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2405.238731] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2405.301761] Lustre: lustre-MDT0000: new disk, initializing [ 2405.388830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2409.388562] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2421.813592] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2421.955319] Lustre: 104454:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2422.221847] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2427.585489] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2434.949099] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2438.707779] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2445.765943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2451.346534] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2452.173225] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2459.331192] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2465.019055] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2465.348089] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2471.634936] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2471.801757] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2478.048429] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2478.051362] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2478.055166] Lustre: Skipped 5 previous similar messages [ 2478.055307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2478.073779] Lustre: Skipped 1 previous similar message [ 2479.865509] Lustre: server umount lustre-OST0000 complete [ 2490.152297] Lustre: server umount lustre-MDT0000 complete [ 2493.411789] LustreError: 104462:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2493.422693] LustreError: 104462:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 2493.667218] LustreError: 104447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775849056 with bad export cookie 4771143048540505137 [ 2493.683619] LustreError: 104447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2493.686185] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2494.130499] Lustre: server umount lustre-MDT0001 complete [ 2502.603640] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 15:24:23 (1775849063) [ 2512.758281] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 2515.821256] Key type lgssc unregistered [ 2516.092324] LNet: 108521:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2516.106460] LNetError: 108521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2516.131199] LNet: Removed LNI 192.168.202.127@tcp [ 2516.994435] Key type .llcrypt unregistered [ 2516.996071] Key type ._llcrypt unregistered [ 2537.983371] Key type ._llcrypt registered [ 2537.984835] Key type .llcrypt registered [ 2538.076755] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 2553.015193] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 2554.535244] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2554.578390] alg: No test for adler32 (adler32-zlib) [ 2555.593976] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 2555.845563] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 2557.519227] Key type lgssc registered [ 2559.013728] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2602.447499] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2602.483365] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2603.769249] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2603.797817] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2603.856098] Lustre: lustre-MDT0000: new disk, initializing [ 2603.938459] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2603.966981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2608.705879] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2618.972825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2619.188540] Lustre: lustre-OST0000: new disk, initializing [ 2619.200818] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2619.213672] Lustre: Skipped 1 previous similar message [ 2619.285991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2626.166361] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2626.179469] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2626.203383] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2626.897757] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2637.664841] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2637.888835] Lustre: lustre-OST0001: new disk, initializing [ 2637.902093] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2637.986196] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2644.692214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2644.705738] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2644.778378] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2645.313400] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2655.204054] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2655.224562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2655.228131] Lustre: Skipped 1 previous similar message [ 2660.321420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2660.329138] Lustre: Skipped 1 previous similar message [ 2660.874452] Lustre: server umount lustre-MDT0000 complete [ 2668.374984] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2672.846457] LustreError: 113498:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775849235 with bad export cookie 1759030645426473884 [ 2672.847781] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2672.859799] LustreError: 113498:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2672.997071] Lustre: server umount lustre-OST0000 complete [ 2676.931165] Lustre: server umount lustre-OST0001 complete [ 2686.094796] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 15:27:26 (1775849246) [ 2701.821991] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 2730.908930] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 15:28:11 (1775849291) [ 2771.080344] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 2780.653872] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 2817.618129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2817.940868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2817.987723] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2818.092547] Lustre: lustre-MDT0000: new disk, initializing [ 2818.208365] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2818.226862] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2822.994526] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2832.940455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2833.095414] Lustre: 120213:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2833.183864] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2833.188681] Lustre: Skipped 1 previous similar message [ 2833.203084] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 2833.271629] Lustre: lustre-MDT0001: new disk, initializing [ 2833.359924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2833.390454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2833.399900] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2837.893650] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2845.257872] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2848.569465] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2854.212217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2854.459567] Lustre: lustre-OST0000: new disk, initializing [ 2854.462875] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2854.529870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2860.432334] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2864.215429] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2864.229282] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2864.420400] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2867.985287] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2873.317771] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2873.593226] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2878.808603] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2879.009610] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2885.090664] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2885.103248] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2885.115146] LustreError: Skipped 1 previous similar message [ 2885.116465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2885.176442] Lustre: Skipped 2 previous similar messages [ 2890.215405] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2890.221162] Lustre: Skipped 1 previous similar message [ 2895.331387] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2895.611184] Lustre: server umount lustre-OST0000 complete [ 2905.057093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2905.067099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2905.086892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2905.089849] Lustre: Skipped 3 previous similar messages [ 2907.971677] Lustre: server umount lustre-MDT0000 complete [ 2912.620533] LustreError: 120206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775849475 with bad export cookie 1759030645426475102 [ 2912.626946] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2912.630912] LustreError: 120206:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2912.903840] Lustre: server umount lustre-MDT0001 complete [ 2922.244706] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 15:31:22 (1775849482) [ 2930.245588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2930.812703] LustreError: 124221:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2930.829521] LustreError: 124221:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2930.903291] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2936.298969] LustreError: 124222:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2937.158642] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2940.397679] LustreError: 124221:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2945.512515] LustreError: 124222:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2946.577807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2951.572918] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2957.849596] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2961.994730] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2970.716403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2971.177814] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2971.188319] Lustre: Skipped 1 previous similar message [ 2977.947484] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 2985.018425] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2989.984979] Lustre: *** cfs_fail_loc=159, val=0*** [ 3005.652263] Lustre: lustre-MDT0000: Client b3aeac9c-1104-46a4-a0f5-ce2c59a1813a (at 192.168.202.27@tcp) reconnecting [ 3007.299933] Lustre: *** cfs_fail_loc=159, val=0*** [ 3007.305967] Lustre: Skipped 6 previous similar messages [ 3023.601242] Lustre: lustre-MDT0000: Client b3aeac9c-1104-46a4-a0f5-ce2c59a1813a (at 192.168.202.27@tcp) reconnecting [ 3032.553523] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3032.568935] Lustre: Skipped 1 previous similar message [ 3032.574888] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3032.579847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3032.591121] Lustre: Skipped 1 previous similar message [ 3035.462861] Lustre: server umount lustre-OST0000 complete [ 3039.205731] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3039.212068] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3039.229531] Lustre: Skipped 1 previous similar message [ 3045.511303] Lustre: server umount lustre-MDT0000 complete [ 3047.905618] LustreError: 125193:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3047.927757] LustreError: 125193:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3049.141883] LustreError: 124202:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775849611 with bad export cookie 1759030645426475991 [ 3049.144122] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3049.152034] LustreError: 124202:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3049.425284] Lustre: server umount lustre-MDT0001 complete [ 3060.672746] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 3063.560477] Key type lgssc unregistered [ 3063.889712] LNet: 127642:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3063.894848] LNetError: 127642:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3063.912802] LNet: Removed LNI 192.168.202.127@tcp [ 3064.773141] Key type .llcrypt unregistered [ 3064.776492] Key type ._llcrypt unregistered [ 3088.637136] Key type ._llcrypt registered [ 3088.648706] Key type .llcrypt registered [ 3088.796395] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 3089.884520] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3089.974831] alg: No test for adler32 (adler32-zlib) [ 3091.035206] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 3091.229505] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 3092.839513] Key type lgssc registered [ 3094.667546] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3103.302052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3103.325950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3104.869276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3109.510903] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3114.977426] LustreError: 129040:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3114.990286] LustreError: 129040:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3118.886986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3119.438832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3123.981702] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3131.513239] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3136.594855] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3145.158299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3145.632584] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3149.755735] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3152.285214] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3158.241865] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3171.331229] Lustre: *** cfs_fail_loc=159, val=0*** [ 3186.928777] Lustre: lustre-MDT0000: Client 116bac1d-923d-43fc-892b-0f8cc8ff15df (at 192.168.202.27@tcp) reconnecting [ 3190.023550] Lustre: *** cfs_fail_loc=159, val=0*** [ 3206.435298] Lustre: lustre-MDT0001: Client 116bac1d-923d-43fc-892b-0f8cc8ff15df (at 192.168.202.27@tcp) reconnecting [ 3208.381858] Lustre: *** cfs_fail_loc=159, val=0*** [ 3208.389988] Lustre: Skipped 3 previous similar messages [ 3223.786575] Lustre: lustre-MDT0001: Client 116bac1d-923d-43fc-892b-0f8cc8ff15df (at 192.168.202.27@tcp) reconnecting [ 3228.312467] Lustre: 131742:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3231.218216] Lustre: *** cfs_fail_loc=159, val=0*** [ 3231.221987] Lustre: Skipped 3 previous similar messages [ 3246.845430] Lustre: lustre-MDT0000: Client d7f1b853-f934-4e43-9895-2f0571c7d900 (at 192.168.202.27@tcp) reconnecting [ 3248.959742] Lustre: *** cfs_fail_loc=159, val=0*** [ 3248.978547] Lustre: Skipped 21 previous similar messages [ 3265.261069] Lustre: lustre-MDT0000: Client d7f1b853-f934-4e43-9895-2f0571c7d900 (at 192.168.202.27@tcp) reconnecting [ 3266.895779] Lustre: 132034:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3272.672235] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3272.679385] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3272.692567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3273.206915] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3273.215926] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3273.235102] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3276.768329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3276.779775] Lustre: Skipped 1 previous similar message [ 3278.934398] Lustre: server umount lustre-OST0000 complete [ 3287.009313] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3287.033213] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3290.974618] Lustre: server umount lustre-MDT0000 complete [ 3292.128325] LustreError: 129041:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3292.159265] LustreError: 129041:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3295.966968] LustreError: 129021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775849858 with bad export cookie 11972680195746778255 [ 3295.975129] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3295.986316] LustreError: 129021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3296.368895] Lustre: server umount lustre-MDT0001 complete [ 3307.780665] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 3310.767912] Key type lgssc unregistered [ 3311.128455] LNet: 133069:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3311.158609] LNetError: 133069:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3312.230548] LNet: Removed LNI 192.168.202.127@tcp [ 3313.510467] Key type .llcrypt unregistered [ 3313.514514] Key type ._llcrypt unregistered [ 3337.525951] Key type ._llcrypt registered [ 3337.529914] Key type .llcrypt registered [ 3337.660616] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 3339.088532] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3339.112239] alg: No test for adler32 (adler32-zlib) [ 3340.270754] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 3340.498455] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 3342.178844] Key type lgssc registered [ 3343.132966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3350.761580] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3350.780438] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3352.196228] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3356.786044] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3361.774926] LustreError: 134471:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3361.797308] LustreError: 134471:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3364.513699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3364.782586] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3368.828319] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3374.068709] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3377.326518] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3383.328827] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3383.602480] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3388.117266] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3389.751637] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3392.688813] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3396.926119] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3405.452355] Lustre: 136728:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3409.364134] Lustre: 136776:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3411.373094] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3412.839721] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3418.995942] Lustre: 136924:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3420.382884] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3421.811205] Lustre: 137032:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3427.812014] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3427.816378] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3427.818135] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3427.840438] Lustre: Skipped 1 previous similar message [ 3432.407604] Lustre: server umount lustre-OST0000 complete [ 3436.513104] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3436.523156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3436.532289] Lustre: Skipped 1 previous similar message [ 3436.538848] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3438.054358] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3438.063048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3442.441726] Lustre: server umount lustre-MDT0000 complete [ 3443.169406] LustreError: 134465:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3443.210452] LustreError: 134465:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3445.990195] LustreError: 136825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775850008 with bad export cookie 5373585908690978280 [ 3446.004370] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3446.022753] LustreError: 136825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3446.683667] Lustre: server umount lustre-MDT0001 complete [ 3456.996486] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 3460.090277] Key type lgssc unregistered [ 3460.343578] LNet: 138075:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3460.360492] LNetError: 138075:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3461.418997] LNet: Removed LNI 192.168.202.127@tcp [ 3462.443219] Key type .llcrypt unregistered [ 3462.444983] Key type ._llcrypt unregistered [ 3488.319484] Key type ._llcrypt registered [ 3488.322397] Key type .llcrypt registered [ 3488.438378] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 3489.715234] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3489.787022] alg: No test for adler32 (adler32-zlib) [ 3490.858541] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 3491.096576] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 3492.823593] Key type lgssc registered [ 3494.188976] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3502.654883] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3502.697433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3504.283419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3509.073860] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3513.830383] LustreError: 139474:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3513.847035] LustreError: 139474:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3517.123048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3517.515426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3522.220802] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3529.319610] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3532.867634] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3539.333125] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3539.589222] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3542.725615] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3545.728261] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3548.856713] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3552.146742] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3559.154499] Lustre: *** cfs_fail_loc=159, val=0*** [ 3575.511886] Lustre: lustre-MDT0000: Client 35c4b7f0-8b79-43c1-825b-1ae9ed15e448 (at 192.168.202.27@tcp) reconnecting [ 3579.873243] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3579.888845] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3579.891444] Lustre: Skipped 1 previous similar message [ 3579.904305] Lustre: Skipped 1 previous similar message [ 3582.066811] Lustre: server umount lustre-OST0000 complete [ 3589.088744] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3589.097790] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3589.124256] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3590.113931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3590.133139] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3590.138842] Lustre: Skipped 1 previous similar message [ 3592.703955] Lustre: server umount lustre-MDT0000 complete [ 3595.232106] LustreError: 141686:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3595.276168] LustreError: 141686:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3597.734148] LustreError: 139455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775850160 with bad export cookie 4588551966846699034 [ 3597.748491] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3598.083635] Lustre: server umount lustre-MDT0001 complete [ 3611.371444] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 3614.221614] Key type lgssc unregistered [ 3614.543785] LNet: 142784:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3614.549849] LNetError: 142784:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3614.560214] LNet: Removed LNI 192.168.202.127@tcp [ 3615.265338] Key type .llcrypt unregistered [ 3615.282369] Key type ._llcrypt unregistered [ 3638.917533] Key type ._llcrypt registered [ 3638.919992] Key type .llcrypt registered [ 3639.044869] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 3640.055751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3640.201190] alg: No test for adler32 (adler32-zlib) [ 3641.413692] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 3641.681874] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 3643.399139] Key type lgssc registered [ 3644.438819] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3651.678084] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3651.702825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3653.393536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3658.581893] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3663.842714] LustreError: 144182:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3663.860713] LustreError: 144182:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3667.111771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3667.431606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3671.987779] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3677.840618] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3681.570394] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3689.221190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3689.606450] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3692.760717] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 3695.814772] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3698.943329] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 3702.691985] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3723.720420] Lustre: lustre-MDT0001: evicting 76bd6de3-c224-46e8-84a1-b4e8a4e701cf (at 192.168.202.27@tcp) by administrative request [ 3754.467588] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3754.481338] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3754.498301] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3755.491396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3755.492287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3760.612051] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3760.617674] Lustre: Skipped 2 previous similar messages [ 3765.753510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3765.762197] Lustre: Skipped 1 previous similar message [ 3767.263190] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3767.445781] Lustre: server umount lustre-OST0000 complete [ 3774.947060] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3774.953803] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3774.961892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3777.934489] Lustre: server umount lustre-MDT0000 complete [ 3781.091938] LustreError: 144183:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3781.107947] LustreError: 144183:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3781.664885] LustreError: 144164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775850344 with bad export cookie 4754390273900701056 [ 3781.673638] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3781.676680] LustreError: 144164:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3782.217934] Lustre: server umount lustre-MDT0001 complete [ 3792.998598] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 3796.187580] Key type lgssc unregistered [ 3796.511605] LNet: 147898:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3796.519575] LNetError: 147898:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3796.541374] LNet: Removed LNI 192.168.202.127@tcp [ 3797.989941] Key type .llcrypt unregistered [ 3797.994131] Key type ._llcrypt unregistered [ 3883.123673] Key type ._llcrypt registered [ 3883.125394] Key type .llcrypt registered [ 3883.236919] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 3884.887387] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3884.946927] alg: No test for adler32 (adler32-zlib) [ 3885.978719] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 3886.164054] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 3887.856378] Key type lgssc registered [ 3888.934432] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3896.219593] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3896.233829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3897.879634] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3902.045736] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3908.092223] LustreError: 150373:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3908.106986] LustreError: 150373:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3911.015896] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3911.528664] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3916.080118] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3922.974355] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3927.384301] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3934.652211] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3935.277373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3937.326139] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 3941.814358] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 3942.459166] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 3948.427741] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3957.732308] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3957.739845] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3957.748900] LustreError: Skipped 1 previous similar message [ 3957.756029] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3957.776299] Lustre: Skipped 1 previous similar message [ 3962.859132] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3962.872586] Lustre: Skipped 1 previous similar message [ 3964.046069] Lustre: server umount lustre-OST0000 complete [ 3967.968734] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3967.978084] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3967.987034] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3967.996626] Lustre: Skipped 1 previous similar message [ 3968.001429] Lustre: Skipped 1 previous similar message [ 3973.104216] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3973.113215] Lustre: Skipped 2 previous similar messages [ 3978.208925] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3978.213988] Lustre: Skipped 1 previous similar message [ 3982.303169] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3982.574439] Lustre: server umount lustre-MDT0000 complete [ 3983.328716] LustreError: 150368:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3983.359799] LustreError: 150368:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3987.429865] LustreError: 150366:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3987.452108] LustreError: 150366:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3987.593524] LustreError: 150354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775850550 with bad export cookie 17364850120399624486 [ 3987.603785] LustreError: 150354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3987.605748] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3988.356437] Lustre: server umount lustre-MDT0001 complete [ 4001.179990] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 4004.445621] Key type lgssc unregistered [ 4004.660750] LNet: 153563:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4004.669590] LNetError: 153563:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4004.688322] LNet: Removed LNI 192.168.202.127@tcp [ 4005.378703] Key type .llcrypt unregistered [ 4005.380847] Key type ._llcrypt unregistered [ 4037.212230] Key type ._llcrypt registered [ 4037.216313] Key type .llcrypt registered [ 4037.438444] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 4039.222908] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4039.250183] alg: No test for adler32 (adler32-zlib) [ 4040.551093] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 4040.921743] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 4042.623175] Key type lgssc registered [ 4044.568608] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4061.784220] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:50:22 (1775850622) [ 4078.100678] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 4087.303536] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 4126.676865] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4126.709358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4128.009805] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4128.056406] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4128.175627] Lustre: lustre-MDT0000: new disk, initializing [ 4128.311446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4128.339950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4134.807460] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4146.602492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4146.838588] Lustre: 157787:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4146.985302] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4146.995978] Lustre: Skipped 1 previous similar message [ 4147.075610] Lustre: lustre-MDT0001: new disk, initializing [ 4147.189074] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4147.219089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4147.225675] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4151.924968] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4160.203477] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4164.260441] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4172.438731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4172.807388] Lustre: lustre-OST0000: new disk, initializing [ 4172.819179] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4172.922797] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4179.865297] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4183.133256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4183.153735] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4183.327064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4188.686311] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4198.883669] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4198.893983] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4198.896698] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4198.904504] Lustre: Skipped 1 previous similar message [ 4204.006661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4204.027869] Lustre: Skipped 1 previous similar message [ 4204.861269] Lustre: server umount lustre-OST0000 complete [ 4208.612679] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4208.631370] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4208.637866] Lustre: Skipped 1 previous similar message [ 4208.642095] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4212.193294] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4212.199713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4215.107114] Lustre: server umount lustre-MDT0000 complete [ 4219.037784] LustreError: 159530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775850781 with bad export cookie 2805833055869626873 [ 4219.042985] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4219.048762] LustreError: 159530:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4219.514296] Lustre: server umount lustre-MDT0001 complete [ 4231.901469] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 4234.945874] Key type lgssc unregistered [ 4235.273679] LNet: 161304:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4235.281964] LNetError: 161304:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4235.300697] LNet: Removed LNI 192.168.202.127@tcp [ 4236.179121] Key type .llcrypt unregistered [ 4236.182465] Key type ._llcrypt unregistered [ 4264.020582] Key type ._llcrypt registered [ 4264.023488] Key type .llcrypt registered [ 4264.128845] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 4265.327972] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4265.555286] alg: No test for adler32 (adler32-zlib) [ 4266.531748] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 4266.699597] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 4268.375194] Key type lgssc registered [ 4269.555569] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4278.496914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4278.518249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4280.298295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4285.164912] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4289.506543] LustreError: 162704:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4289.533546] LustreError: 162704:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4294.626189] LustreError: 162705:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4296.046497] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4296.480546] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4301.829490] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4307.739284] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4312.474705] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4320.664560] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4321.189167] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4324.365248] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4328.860001] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 4335.796366] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4344.476829] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4344.508409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4344.539695] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4344.560466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4344.590243] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4345.845728] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4345.870795] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4345.884296] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4345.897507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4345.906449] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4346.968237] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4346.985374] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4346.993751] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4346.999774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4347.011140] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4347.914358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.015576] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4349.022349] Lustre: Skipped 1 previous similar message [ 4349.049305] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4349.063180] Lustre: Skipped 1 previous similar message [ 4349.069504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4349.072282] Lustre: Skipped 1 previous similar message [ 4349.077396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4349.117238] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4349.128243] Lustre: Skipped 1 previous similar message [ 4350.147084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4351.181792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4352.282392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.252334] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4353.255462] Lustre: Skipped 3 previous similar messages [ 4353.268411] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4353.276186] Lustre: Skipped 3 previous similar messages [ 4353.280723] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4353.288367] Lustre: Skipped 3 previous similar messages [ 4353.292537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4353.302872] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4353.311330] Lustre: Skipped 3 previous similar messages [ 4354.466917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4355.299526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4356.405309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4357.434413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4358.300499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4359.360892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4360.337216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.343399] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4361.350758] Lustre: Skipped 7 previous similar messages [ 4361.367633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4361.376618] Lustre: Skipped 7 previous similar messages [ 4361.381596] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4361.387293] Lustre: Skipped 7 previous similar messages [ 4361.391812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4361.406526] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4361.419927] Lustre: Skipped 7 previous similar messages [ 4362.175736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4362.960573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4363.884966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4364.945368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4365.705066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4366.605397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4367.641321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4368.776468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4369.993903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4371.386974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4372.664239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4373.658878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4374.612530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4375.575431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4376.720801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4377.919429] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4377.926214] Lustre: Skipped 15 previous similar messages [ 4377.959592] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4377.974065] Lustre: Skipped 15 previous similar messages [ 4377.980771] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4377.989420] Lustre: Skipped 15 previous similar messages [ 4377.999887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4378.021913] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4378.024694] Lustre: Skipped 15 previous similar messages [ 4378.975483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4379.901986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4380.997269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4381.971477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4382.753707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4383.789814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4384.941364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4385.840423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4386.903334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4387.860649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4388.953092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4389.798747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4390.693312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4391.738604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4392.882985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4393.879994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4394.809383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4395.760336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4396.913832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4397.973609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4399.021559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.130200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4400.982157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4401.947584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4402.935830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4403.737426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4404.608955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4405.532881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4406.509362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4407.446941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4408.488807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4409.783072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.768182] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4410.776142] Lustre: Skipped 32 previous similar messages [ 4410.799743] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4410.811738] Lustre: Skipped 32 previous similar messages [ 4410.815924] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4410.821279] Lustre: Skipped 32 previous similar messages [ 4410.835956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4410.848825] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4410.860601] Lustre: Skipped 32 previous similar messages [ 4411.874692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4412.712568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4413.894815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4415.201503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4416.217509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4417.282815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4418.202906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4419.311716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4420.173374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4421.184236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4422.224276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.047531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4423.916660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4424.894527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4426.053742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4427.185191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4428.281264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4429.229228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4430.105585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4431.107575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4432.287663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.153835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4433.965084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4434.905934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4435.948409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4436.614752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4437.596773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4438.536910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4439.535892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4440.834756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4441.581723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4442.625626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4443.974597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4445.142858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4446.256453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4447.587283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4448.660855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4449.720757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4450.593618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4451.758705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4452.618925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4453.443918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4454.241428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4455.323348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4456.702680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4457.360105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4458.436689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4459.347184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4460.492346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4461.494539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4462.732455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4464.043182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4465.222323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4466.385676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4467.359709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.133801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4468.738615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4469.518623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4470.771776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4471.603047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4472.509527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4473.678978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4474.714264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.392814] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4475.398080] Lustre: Skipped 63 previous similar messages [ 4475.410642] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4475.428174] Lustre: Skipped 63 previous similar messages [ 4475.432456] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4475.435580] Lustre: Skipped 63 previous similar messages [ 4475.440691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4475.453463] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4475.461596] Lustre: Skipped 63 previous similar messages [ 4476.335415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4477.566554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4478.642077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4479.728528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4480.551112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4481.520622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4482.533073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4483.633565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4484.606845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4485.554238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4486.780563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4487.951029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4488.944053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4489.836885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4490.824173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4491.954235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4492.752937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4493.738984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4494.515575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4495.454516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4496.271253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4497.244777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.138068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4498.990376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4499.735869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4500.710779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4501.630606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4502.776721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4503.730707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4504.686692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4505.582759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4506.621618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4507.576080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4508.465953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4509.379585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4510.355078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4511.257109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4512.076371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4513.053079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.147120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4514.951023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4515.749272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4516.515462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4517.544932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4518.470777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4519.430377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4520.448582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4521.609874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4522.327395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4523.404048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4524.197801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4525.031789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4526.147239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4527.288526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4528.328456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4529.576489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4530.698360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4531.812966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4532.592856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4533.860746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4535.072497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.054502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4536.764503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4537.895458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4538.754969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4539.686564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4540.652243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4541.691383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4542.607979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4543.571836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4544.578849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4545.620043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4546.559321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4547.655714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4548.589245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4549.792755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4550.561365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4551.580462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4552.808670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4553.989666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4555.273099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4556.272714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4557.322797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4558.220340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4559.274354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4560.284152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4561.317614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4562.130463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4563.358786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4564.520758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4566.011541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4567.118232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4568.097626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4569.319471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4570.420020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4571.701051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4572.602512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4573.357811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4574.507655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4576.168917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4577.502061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4578.822099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4579.780278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4581.283157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4582.057164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4583.468097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4585.102479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4586.868125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4588.142806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4589.529120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4591.431520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4592.763251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4594.058996] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4594.080398] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4594.094446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4595.538737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4596.731392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4598.029759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4599.138493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4600.081744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4601.260991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4602.329910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4603.200620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.287418] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4604.296351] Lustre: Skipped 121 previous similar messages [ 4604.333996] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4604.344509] Lustre: Skipped 121 previous similar messages [ 4604.348703] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4604.354914] Lustre: Skipped 121 previous similar messages [ 4604.359719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4604.390668] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4604.394894] Lustre: Skipped 121 previous similar messages [ 4605.434391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4606.365961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4607.448943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4608.743567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4609.964940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4610.971976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4611.785755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4612.719837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.949245] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4613.956423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4613.956897] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4615.415664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4616.216317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4617.178677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4618.071604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.014069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4619.748120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4620.828446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4621.819209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4622.734058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4623.673327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4624.677897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4625.494933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4626.543308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4627.708359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4628.779076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4629.492413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4630.445485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4631.388793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4632.150836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4633.049365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.002074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4634.922932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4635.830587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4636.657551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4638.195089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4639.533428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4640.706510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4641.851285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4643.084418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4644.052622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.233605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4645.979770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.138250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4647.867690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4649.030567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4650.138572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4651.150510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4652.405862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4653.269570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4654.409511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4655.621559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4656.750981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4657.936702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4658.713576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4659.523075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4660.328079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4661.355157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4662.613267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4663.662938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4664.908560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4665.971641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4666.989074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4668.198354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.044602] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4669.065975] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4669.098053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4669.935368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4671.123368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4672.174644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4673.268721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.391724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4675.510070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.391196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4677.643409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.926962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.106320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.228529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.304856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.451261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.695865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.778075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4686.966112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.312632] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4688.322798] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4688.359574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.440617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.522811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.464024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.538572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.722542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.575396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.665310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.964505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.033627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.309966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.444804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.387625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.554949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.736401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.732890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.853604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.004201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.295511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.443097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.667248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4711.651811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.811645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.787621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.985610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.985469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.339883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4718.749976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.474561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.292147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.241593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.613130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.447061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.434975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.810055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.256608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4729.150041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.289645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4731.519206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.765459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4734.125441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.341528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.171628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.920271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.320219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.680381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.658871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.943450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4744.159684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.460767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.781530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.962040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.918664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.000147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.048179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.047604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.271654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.811895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.790716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.675518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.551502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.942616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.880645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.682553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.850606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.563261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.597679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.676993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.520869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.740894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.574209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.353369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.131716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.228777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.613544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.024325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.895166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.285870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.979537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.016078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.037714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.072620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.956244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.161799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.137748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.919285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.822188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.658393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.678179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.895278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.800570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.725597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.475272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.540332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.389080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.582183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.285390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.162481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.944464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.876817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.631082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.520740] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4797.527968] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4797.541976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.598698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.618841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.433678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.229666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.061370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.004940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.764793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.610855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.424569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.448075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.450069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.547192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.520890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.900802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.858845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.129724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.480598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.592527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.541624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.809831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.562337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.584573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.528938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.797132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.971620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.117677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.463696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.845888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.013489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.858707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.925557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.782209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.136161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.488635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.552590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.487025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.981075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.122377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.586080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.799209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.951521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.139182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.267716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.468091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.408733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.488322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.382895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.477578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.449640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.545530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.530444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.392294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.373233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.177391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.081342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.928906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.938153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.014413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.986370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.215586] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4861.222432] Lustre: Skipped 240 previous similar messages [ 4861.246987] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4861.254259] Lustre: Skipped 240 previous similar messages [ 4861.257914] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4861.262374] Lustre: Skipped 240 previous similar messages [ 4861.269460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.276144] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4861.279218] Lustre: Skipped 240 previous similar messages [ 4862.141164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.895626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.889831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.148788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.090621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.259530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.176733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.267756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.441821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.507190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.713429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.836813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.963020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.204218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.208839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.432714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.743915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.757322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.858274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.903968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.735643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.851978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.105872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.128405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.349805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.707815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.751530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.980838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.915940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.992487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.977624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.058840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.205491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.184994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.459937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.393953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.472471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.345318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.398289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.391266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.635499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.553070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.527952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.809969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.031560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.347649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.700421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.809498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.851308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.953107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.967239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.155764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.392823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.705340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.584834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.810865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.011102] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4924.022525] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4924.031960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.923232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.106904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.280641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.548790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.802598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.595666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.636179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.669891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.810452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.549902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.980570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.947408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.290140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.511946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.905349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.176659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.614852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.812399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.160763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.232239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.539176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.888081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.145774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.576157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.651232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.025986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.510500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.910842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.522090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.612298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.801525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.893604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.131981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.993814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.167440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.995585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.991726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.883439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.939367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.742087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.000874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.819935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.677991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.762244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.953375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.051531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.979710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.030535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.100438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.294243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.459612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.505436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.384131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.166414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.152231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.047382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.132752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.337970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.224349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.251926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.364406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.245314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.322256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.230871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.478851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.518515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.774860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.222377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.544705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.453981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.182617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.375814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.724658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.090561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.276230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.995740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.233367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.998325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.303570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.415384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.740944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.915976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.801992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.893380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.868310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.050229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.565898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.081075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.608271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.997609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.416289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.533948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.023456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.929831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.138752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.718664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.822247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.757824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.204990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.576175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.141635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.187179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.505097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.793587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.268832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.145985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.212112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.333274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.585721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.504379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.802139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.073481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.356982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.267966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.546962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.507562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.574659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.676428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.625607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.831576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.889470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.732657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.655334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.712068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.547498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.533544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.509700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.640823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.634897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.683827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.033210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.318238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.506609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.558244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.689746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.839683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.902682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.881397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.946109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.229466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.305371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.519394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.527853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.602763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.691757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.684772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.710481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.664305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.856187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.704920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.866254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.944237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.995519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.952644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.869355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.770629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.743507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.828095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.785263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.073188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.391996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.501985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.839239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.934017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.114127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.329556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.354522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.268372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.036340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.351518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.562610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.497423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.521254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.760496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.084709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.204393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.949346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.034285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.519336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.982454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.027848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.984525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.433294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.576453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.770278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.102269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.339461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.721288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.733080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.597836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.648969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.609191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.526946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.599501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.644071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.884788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.069423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.776938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.891845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.931550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.226262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.189538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.330704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.292695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.445967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.489166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.449367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.348520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.383597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.351181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.380897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.250707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.261048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.087376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.090857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.913883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.727690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.717116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.762986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.111131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.339783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.726396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.096305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.325389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.449345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.557112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.555261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.558987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.534110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.617754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.445531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.567498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.514513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.555414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.634870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.623366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.778054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.871236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.742924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.762470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.819768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.617284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.661587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.384320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.461191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.379928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.408540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.450217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.443165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.534499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.487935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.526863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.771884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.985687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.047066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.270433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.368086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.414847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.583440] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5213.600519] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 5213.609414] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5213.611052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.619478] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) Skipped 1 previous similar message [ 5214.732161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.751437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.903755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.930601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.803211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.739901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.656071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.780242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.699734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.097491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.861153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.301732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.355459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.383254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.472449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.691209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.887894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.096756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.624435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.955908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.825903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.721176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.686713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.935900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.709383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.777617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.743306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.581020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.848265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.105883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.086657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.025521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.791040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.117257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.368859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.057483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.041690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.121399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.369946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.234577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.836175] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5257.841528] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5257.847993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.904530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.189810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.325081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.728971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.068067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.439452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.706631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.801169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.745509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.996796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.268054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.157966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.332496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.354846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.459187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.429838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.577949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.500768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.548483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.706460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.835676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.951278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.967806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.011542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.074529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.207836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.163816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.298296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.260400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.434475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.641241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.825956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.118035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.423695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.449642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.611865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.867562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.072956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.046867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.088075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.007982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.976476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.748508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.690673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.656405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.719918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.792288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.680679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.761729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.789618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.993206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.407909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.683308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.976868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.153564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.476528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.431600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.620375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.974976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.294524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.654423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.043278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.228492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.321652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.170263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.351127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.514325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.619023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.761628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.083581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.886361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.849674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.046409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.283117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.582481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.250124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.808961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.197814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.421655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.599821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.034502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.134589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.970375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.005130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.040835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.317475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.367172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.323866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.273370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.471527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.847468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.001813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.106701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.213266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.419036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.494036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.275383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.166856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.048986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.969475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.914703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.539334] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5373.546746] Lustre: Skipped 458 previous similar messages [ 5373.574968] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5373.591486] Lustre: Skipped 458 previous similar messages [ 5373.594889] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5373.600491] Lustre: Skipped 458 previous similar messages [ 5373.602957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.610224] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5373.623923] Lustre: Skipped 458 previous similar messages [ 5374.790504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.834417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.765236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.758564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.772823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.786729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.956402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.841385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.910452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.016888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.023570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.982834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.803434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.872691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.838951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.081242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.016129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.057932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.923483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.862832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.915866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.919889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.032257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.132246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.115981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.311650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.388664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.372842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.302271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.420808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.201077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.239540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.251554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.313387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.379048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.605207] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5410.616811] LustreError: 162724:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5410.630994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.664916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.558836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.444538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.477332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.311524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.303277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.488091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.331425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.444913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.966531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.269513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.768880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.534331] LustreError: 164301:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5870.545546] LustreError: 162724:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6619.118288] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6619.148744] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6619.151810] Lustre: Skipped 48 previous similar messages [ 6619.161707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6619.195462] Lustre: Skipped 1 previous similar message [ 6620.188865] Lustre: server umount lustre-OST0000 complete [ 6626.275784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6626.317690] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6629.350263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6629.362658] Lustre: Skipped 1 previous similar message [ 6632.604335] Lustre: server umount lustre-MDT0000 complete [ 6634.466775] LustreError: 162701:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6634.497959] LustreError: 162701:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6638.531030] LustreError: 162685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775853201 with bad export cookie 13191170680925918642 [ 6638.541408] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6638.542119] LustreError: 162685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6639.060957] Lustre: server umount lustre-MDT0001 complete [ 6653.217618] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 6656.789861] Key type lgssc unregistered [ 6657.125754] LNet: 217657:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6657.137355] LNetError: 217657:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6657.154759] LNet: Removed LNI 192.168.202.127@tcp [ 6658.219680] Key type .llcrypt unregistered [ 6658.222536] Key type ._llcrypt unregistered [ 6685.804657] Key type ._llcrypt registered [ 6685.807509] Key type .llcrypt registered [ 6685.979731] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 6687.424567] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6687.626771] alg: No test for adler32 (adler32-zlib) [ 6688.984585] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 6689.239196] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 6690.927196] Key type lgssc registered [ 6692.521761] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6702.362422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6702.417085] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6704.082242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6708.948372] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6713.316846] LustreError: 219056:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6713.372672] LustreError: 219056:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6718.436124] LustreError: 219057:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6719.288522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6719.890121] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6725.525823] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6732.919596] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 6737.071601] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 6745.172777] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6745.548845] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6748.673544] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 6752.548651] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6761.221503] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6771.372174] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6774.456464] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6774.490915] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6774.509081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.542033] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6774.542277] LustreError: 219076:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6779.585657] Lustre: lustre-OST0000: Not available for connect from 192.168.202.27@tcp (stopping) [ 6780.393358] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6780.400050] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6780.424984] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6780.441293] Lustre: Skipped 1 previous similar message [ 6783.904768] Lustre: server umount lustre-OST0000 complete [ 6794.642059] LustreError: 219076:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6820.964055] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6821.536243] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6829.205363] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6837.785405] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 6846.063131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.082197] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.089651] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6846.123705] Lustre: Skipped 1 previous similar message [ 6847.418110] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 7 sec [ 6852.358312] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ba6d8e94000.ost_server_uuid 50 [ 6854.295758] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ba6d8e94000.ost_server_uuid in FULL state after 0 sec [ 6861.601230] Lustre: server umount lustre-OST0000 complete [ 6866.913655] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6866.914976] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6866.930329] Lustre: Skipped 2 previous similar messages [ 6866.947049] Lustre: Skipped 1 previous similar message [ 6872.037131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6872.046230] Lustre: Skipped 1 previous similar message [ 6876.129353] LustreError: 219076:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000003080556c x1862117098623104/t0(0) o7->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 264/240 e 0 to 0 dl 0 ref 2 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6876.196905] Lustre: server umount lustre-MDT0000 complete [ 6877.154810] LustreError: 219051:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6877.181873] LustreError: 219051:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6878.175654] Lustre: 218369:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775853424/real 1775853424] req@00000000166d5f04 x1862117098620032/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1775853440 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6878.179961] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6878.193331] Lustre: 218369:0:(client.c:2479:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 6881.035567] LustreError: 219778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775853443 with bad export cookie 9294823095223657118 [ 6881.045581] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6881.049562] LustreError: 219778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6881.777985] Lustre: server umount lustre-MDT0001 complete [ 6898.744458] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 6902.184840] Key type lgssc unregistered [ 6902.452779] LNet: 223724:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6902.458638] LNetError: 223724:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6902.494505] LNet: Removed LNI 192.168.202.127@tcp [ 6903.199748] Key type .llcrypt unregistered [ 6903.203288] Key type ._llcrypt unregistered [ 6942.944538] Key type ._llcrypt registered [ 6942.947213] Key type .llcrypt registered [ 6943.137474] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 6944.769220] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6944.801883] alg: No test for adler32 (adler32-zlib) [ 6946.196109] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 6946.556597] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 6948.335285] Key type lgssc registered [ 6950.212222] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6963.145234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6963.178994] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6965.000190] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6970.851690] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6974.442273] LustreError: 225436:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6974.459590] LustreError: 225436:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6979.553118] LustreError: 225437:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6982.009375] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6982.479528] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6988.196308] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 6991.829339] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7000.539297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7001.067882] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7009.052483] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7010.335222] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 7019.522987] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7019.731648] Lustre: lustre-OST0001: new disk, initializing [ 7019.740045] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7019.939251] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7027.926669] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7028.410085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7028.425273] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7028.523159] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7041.764763] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7049.429238] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7070.351765] Lustre: server umount lustre-MDT0000 complete [ 7074.811604] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7074.822090] LustreError: 225432:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7074.824763] Lustre: Skipped 2 previous similar messages [ 7074.844039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7074.851749] LustreError: 225432:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7075.872451] LustreError: 225419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775853638 with bad export cookie 15301639832825100443 [ 7075.879352] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7075.890608] LustreError: 225419:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7076.987867] Lustre: server umount lustre-MDT0001 complete [ 7091.656420] Lustre: server umount lustre-OST0000 complete [ 7095.775288] Lustre: 224700:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775853642/real 1775853642] req@ffff8c33a767a680 x1862117369150720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775853658 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7095.815211] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7095.823081] Lustre: Skipped 1 previous similar message [ 7097.107833] Lustre: server umount lustre-OST0001 complete [ 7122.987129] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 7137.262880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7137.886727] LustreError: 230772:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7138.012116] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7143.395017] LustreError: 230773:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7143.562144] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7152.617939] LustreError: 230772:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7152.670817] LustreError: 230772:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7154.051539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7154.373219] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7160.061586] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7164.395631] Lustre: 231881:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7174.328903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7175.051170] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7179.133740] LustreError: 232236:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7183.196403] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7183.332626] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49732 to 0x280000401:49761) [ 7193.936744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7200.305583] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7209.595170] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7213.798737] Lustre: 233729:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7235.041453] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7235.052674] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7235.077704] Lustre: Skipped 2 previous similar messages [ 7235.099732] Lustre: Skipped 3 previous similar messages [ 7238.621084] Lustre: server umount mylustre-MDT0000 complete [ 7240.162911] LustreError: 230772:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7240.174086] LustreError: 230772:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 7242.975081] LustreError: 230804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775853805 with bad export cookie 15301639832825103257 [ 7242.981455] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7242.987677] LustreError: 230804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7243.562576] Lustre: server umount mylustre-MDT0001 complete [ 7259.071392] Lustre: server umount mylustre-OST0000 complete [ 7260.383726] Lustre: 224698:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775853807/real 1775853807] req@ffff8c33a4989880 x1862117369221248/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775853823 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7260.435728] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7260.451335] Lustre: Skipped 1 previous similar message [ 7263.711362] Lustre: 224700:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775853810/real 1775853810] req@ffff8c33859fb100 x1862117369221504/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775853826 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7264.233918] Lustre: server umount mylustre-OST0001 complete [ 7289.642022] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 7303.293624] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7304.170175] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7304.182348] Lustre: Skipped 1 previous similar message [ 7309.285518] LustreError: 236023:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7309.301257] LustreError: 236023:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7309.798396] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7320.535771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7326.827857] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7330.878423] Lustre: 237131:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7341.605949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7348.481966] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49793) [ 7350.476504] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7359.763954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7367.607597] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7376.678899] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7381.119500] Lustre: 238978:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7395.811179] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7395.828559] Lustre: Skipped 3 previous similar messages [ 7395.830115] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7395.838179] Lustre: Skipped 1 previous similar message [ 7400.930126] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7400.946018] Lustre: Skipped 2 previous similar messages [ 7401.463211] Lustre: server umount tfs-MDT0000 complete [ 7406.418100] LustreError: 235999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775853969 with bad export cookie 15301639832825104923 [ 7406.422527] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7406.432508] LustreError: 235999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7406.746826] Lustre: server umount tfs-MDT0001 complete [ 7422.925189] Lustre: server umount tfs-OST0000 complete [ 7426.532287] Lustre: 224700:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775853973/real 1775853973] req@ffff8c34b7612a00 x1862117369283200/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775853989 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7426.561536] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7427.546621] Lustre: server umount tfs-OST0001 complete [ 7453.074587] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 7465.492036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7466.206391] LustreError: 241212:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7466.229560] LustreError: 241212:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 7466.384095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7466.386965] Lustre: Skipped 3 previous similar messages [ 7471.625053] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7480.672901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7485.287691] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7489.111468] Lustre: 242321:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7499.086373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7508.022920] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7510.839059] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49763 to 0x280000401:49825) [ 7517.922466] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7525.488578] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7534.621274] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7538.883859] Lustre: 244165:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7552.544752] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:48:33 (1775854113) [ 7573.473391] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7573.479483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7573.498204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7576.065809] Lustre: server umount lustre-MDT0000 complete [ 7579.541034] LustreError: 241189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775854142 with bad export cookie 15301639832825106365 [ 7579.545398] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7579.565447] LustreError: 241189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7579.623215] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7579.637352] Lustre: Skipped 5 previous similar messages [ 7586.288586] Lustre: server umount lustre-MDT0001 complete [ 7590.980939] Lustre: server umount lustre-OST0000 complete [ 7595.340342] Lustre: server umount lustre-OST0001 complete [ 7602.387305] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 7614.997814] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 7657.402481] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7657.402481] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7657.402481] [ 7657.444406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7657.812394] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7657.921979] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7658.099359] Lustre: lustre-MDT0000: new disk, initializing [ 7658.288549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7658.297571] Lustre: Skipped 3 previous similar messages [ 7658.349560] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7663.494465] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7674.421951] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7674.421951] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7674.421951] [ 7674.447541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7674.578930] Lustre: 247317:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7674.648616] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7674.654184] Lustre: Skipped 1 previous similar message [ 7674.743060] Lustre: lustre-MDT0001: new disk, initializing [ 7674.907381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7674.911186] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7679.953694] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7691.414825] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7691.675457] Lustre: lustre-OST0000: new disk, initializing [ 7691.684792] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7693.376720] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7693.389548] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7693.512977] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7699.028839] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7710.206613] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7710.451993] Lustre: lustre-OST0001: new disk, initializing [ 7710.455418] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7711.736316] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7711.754799] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7711.958617] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7717.429846] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7728.818484] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:51:29 (1775854289) [ 7744.333948] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:51:45 (1775854305) [ 7760.354070] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7760.366075] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7760.377687] Lustre: Skipped 5 previous similar messages [ 7760.385290] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7760.391694] Lustre: Skipped 3 previous similar messages [ 7765.697470] Lustre: server umount lustre-OST0000 complete [ 7768.548121] LustreError: 249652:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7768.582779] LustreError: 249652:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 7772.129959] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7772.139915] LustreError: Skipped 1 previous similar message [ 7777.217899] Lustre: server umount lustre-MDT0000 complete [ 7781.596064] LustreError: 247310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775854344 with bad export cookie 15301639832825107688 [ 7781.609602] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7781.615269] LustreError: 247310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7783.915242] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7783.927927] Lustre: Skipped 6 previous similar messages [ 7788.203360] Lustre: server umount lustre-MDT0001 complete [ 7801.148605] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 7813.852848] Lustre: server umount lustre-OST0001 complete [ 7821.059628] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 7836.659668] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 7878.844207] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7879.375791] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7879.444171] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7879.601886] Lustre: lustre-MDT0000: new disk, initializing [ 7879.769399] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7885.496271] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7897.770837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7897.961950] Lustre: 254210:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7898.078383] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7898.083694] Lustre: Skipped 1 previous similar message [ 7898.139598] Lustre: lustre-MDT0001: new disk, initializing [ 7898.250273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7898.258469] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7903.288862] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7912.623594] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7917.609585] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7924.926677] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7925.134172] Lustre: lustre-OST0000: new disk, initializing [ 7925.138606] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7925.243156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7925.247916] Lustre: Skipped 5 previous similar messages [ 7926.969988] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7926.987787] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7927.155645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7932.325667] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 7940.480905] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7952.869196] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7952.886261] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7952.888866] LustreError: Skipped 1 previous similar message [ 7952.923290] Lustre: Skipped 6 previous similar messages [ 7952.947489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7952.953356] Lustre: Skipped 2 previous similar messages [ 7954.161553] Lustre: server umount lustre-OST0000 complete [ 7972.319366] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7972.553961] Lustre: server umount lustre-MDT0000 complete [ 7976.588890] LustreError: 254202:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775854539 with bad export cookie 15301639832825109165 [ 7976.600583] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7976.603786] LustreError: 254202:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7977.004985] Lustre: server umount lustre-MDT0001 complete [ 7989.801804] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 7994.203203] Key type lgssc unregistered [ 7994.548674] LNet: 257587:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7994.569111] LNetError: 257587:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7994.594843] LNet: Removed LNI 192.168.202.127@tcp [ 7995.825238] Key type .llcrypt unregistered [ 7995.827532] Key type ._llcrypt unregistered [ 8025.347921] Key type ._llcrypt registered [ 8025.353585] Key type .llcrypt registered [ 8025.492969] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 8026.764362] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8026.828132] alg: No test for adler32 (adler32-zlib) [ 8027.987759] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 8028.277215] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 8029.935162] Key type lgssc registered [ 8031.231488] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8040.709467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8040.740754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8042.335512] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8047.567804] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8052.710569] LustreError: 259035:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8052.729586] LustreError: 259035:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8056.301744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8056.798055] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8060.829353] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8067.783909] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8071.780272] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8079.146400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8079.654255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8085.082219] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8087.868436] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8091.702668] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8093.489086] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8097.249492] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8097.257743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8097.269639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8098.274830] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8098.317300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8098.349933] Lustre: Skipped 1 previous similar message [ 8103.307816] Lustre: server umount lustre-OST0000 complete [ 8113.123573] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8113.138431] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8113.155786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8118.754802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8118.764925] Lustre: Skipped 2 previous similar messages [ 8123.359976] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8123.624635] Lustre: server umount lustre-MDT0000 complete [ 8123.881882] LustreError: 259746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8123.904199] LustreError: 259746:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8127.473853] LustreError: 259015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775854690 with bad export cookie 16480194991407882082 [ 8127.484194] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8127.486129] LustreError: 259015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8127.852973] Lustre: server umount lustre-MDT0001 complete [ 8139.607682] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 8142.806504] Key type lgssc unregistered [ 8143.113688] LNet: 262270:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8143.125588] LNetError: 262270:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8144.165628] LNet: Removed LNI 192.168.202.127@tcp [ 8145.310282] Key type .llcrypt unregistered [ 8145.314830] Key type ._llcrypt unregistered [ 8164.214727] Key type ._llcrypt registered [ 8164.215920] Key type .llcrypt registered [ 8164.270452] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 8166.084311] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8166.147335] alg: No test for adler32 (adler32-zlib) [ 8167.199320] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 8167.500381] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 8169.191164] Key type lgssc registered [ 8170.431496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8181.788831] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8181.809522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8197.471335] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8202.593401] LustreError: 263447:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.127@tcp: failed processing log, type 4: rc = -110 [ 8229.343176] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8237.025257] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8244.453727] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8247.799353] Lustre: server umount lustre-OST0000 complete [ 8268.479137] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 8271.193960] Key type lgssc unregistered [ 8271.546933] LNet: 264949:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8271.556154] LNetError: 264949:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8271.571842] LNet: Removed LNI 192.168.202.127@tcp [ 8272.605556] Key type .llcrypt unregistered [ 8272.610791] Key type ._llcrypt unregistered [ 8318.592460] Key type ._llcrypt registered [ 8318.595264] Key type .llcrypt registered [ 8318.725085] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 8320.473057] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8320.616672] alg: No test for adler32 (adler32-zlib) [ 8321.900749] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 8322.214177] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 8323.943218] Key type lgssc registered [ 8325.645252] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8340.625781] loop0: detected capacity change from 0 to 204800000 [ 8340.847813] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8340.858225] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8341.181855] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8342.314626] loop0: detected capacity change from 0 to 204800000 [ 8342.585040] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8342.616249] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8343.113427] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8344.350436] loop0: detected capacity change from 0 to 204800000 [ 8344.680258] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8344.699638] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8344.881657] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8345.917135] loop0: detected capacity change from 0 to 204800000 [ 8346.139995] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8346.152344] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8346.282809] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8347.298389] loop0: detected capacity change from 0 to 204800000 [ 8347.323665] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8351.755970] loop0: detected capacity change from 0 to 204800000 [ 8351.788502] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8356.063888] loop0: detected capacity change from 0 to 204800000 [ 8356.103520] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8359.955128] loop0: detected capacity change from 0 to 204800000 [ 8360.025885] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8364.070269] loop0: detected capacity change from 0 to 204800000 [ 8364.156270] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8364.216546] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8364.310742] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8364.317491] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8364.323961] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8364.330356] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8364.340473] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8364.346965] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8364.354293] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8364.360491] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8364.366101] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8364.371836] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8364.376709] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8364.381232] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8364.385733] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8364.390619] Lustre: 268363:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8370.318252] Lustre: server umount MGS complete [ 8371.249604] loop0: detected capacity change from 0 to 204800000 [ 8371.365512] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8371.846518] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8371.931525] Lustre: lustre-MDT0000: Aborting client recovery [ 8372.946539] loop1: detected capacity change from 0 to 204800000 [ 8373.134593] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8373.232457] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8373.239890] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8373.266956] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8373.286422] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8373.294989] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8373.301118] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8373.311425] Lustre: 268796:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8373.324623] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8373.486831] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8373.523810] Lustre: lustre-MDT0001: Aborting client recovery [ 8374.454197] loop2: detected capacity change from 0 to 204800000 [ 8374.639934] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8374.704530] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8374.721468] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8374.738996] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8374.761104] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8374.781965] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8374.794355] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8374.805424] Lustre: 268873:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8374.923407] Lustre: Found index 0 for lustre-OST0000, updating log [ 8375.019564] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8375.043460] Lustre: lustre-OST0000: Aborting recovery [ 8375.832289] loop3: detected capacity change from 0 to 204800000 [ 8375.897830] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8375.995338] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8376.011755] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8376.025389] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8376.039219] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8376.049567] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8376.065770] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8376.085993] Lustre: 268956:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8376.135038] Lustre: Found index 1 for lustre-OST0001, updating log [ 8376.247984] Lustre: lustre-OST0001: Aborting recovery [ 8381.366199] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8381.383766] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8381.402185] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8381.404835] LustreError: 268739:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8381.406813] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8383.421912] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8383.430317] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8384.515445] LustreError: 268705:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 662us: rc = -2 [ 8385.036980] LustreError: 268705:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 596us: rc = -2 [ 8385.053840] LustreError: 268705:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8398.821176] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8398.841167] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8398.849178] Lustre: Skipped 1 previous similar message [ 8398.865095] Lustre: Skipped 3 previous similar messages [ 8403.954932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8403.970651] Lustre: Skipped 3 previous similar messages [ 8404.541373] Lustre: server umount lustre-MDT0000 complete [ 8405.582258] LustreError: 268689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775854968 with bad export cookie 11731514351687298497 [ 8405.585237] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8405.593755] LustreError: 268689:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8409.061895] LustreError: 268703:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8409.068584] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8409.070046] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8409.077341] LustreError: 268703:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8409.087024] Lustre: Skipped 3 previous similar messages [ 8410.080200] LustreError: 268702:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8410.103687] LustreError: 268702:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8412.215654] Lustre: server umount lustre-MDT0001 complete [ 8419.562499] Lustre: server umount lustre-OST0000 complete [ 8435.167144] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8435.431740] Lustre: server umount lustre-OST0001 complete [ 8445.656420] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 17:03:26 (1775855006) [ 8479.355368] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 8490.503585] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 8538.431114] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8538.773609] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8538.808634] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8538.944824] Lustre: lustre-MDT0000: new disk, initializing [ 8539.093591] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8539.100265] Lustre: Skipped 1 previous similar message [ 8539.117290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8544.594660] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8557.095969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8557.268556] Lustre: 273009:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8557.390459] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8557.394550] Lustre: Skipped 1 previous similar message [ 8557.524475] Lustre: lustre-MDT0001: new disk, initializing [ 8557.661935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8557.695811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8557.707097] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8563.304213] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8571.689924] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8576.362535] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8584.331064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8584.799671] Lustre: lustre-OST0000: new disk, initializing [ 8584.805930] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8584.954969] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8591.444198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8591.455149] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8591.603547] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8592.803292] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8602.071879] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8608.328530] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8622.970194] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8627.730045] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8633.050399] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8678.881649] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8678.896058] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8678.920738] Lustre: Skipped 1 previous similar message [ 8678.934554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8678.946313] Lustre: Skipped 4 previous similar messages [ 8684.023942] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8684.034485] Lustre: Skipped 1 previous similar message [ 8684.386713] Lustre: server umount lustre-OST0000 complete [ 8690.656070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8690.665983] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8692.195520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8692.209979] Lustre: Skipped 2 previous similar messages [ 8695.791760] Lustre: server umount lustre-MDT0000 complete [ 8697.312346] LustreError: 273018:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8697.326754] LustreError: 273018:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8700.090858] LustreError: 276146:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775855262 with bad export cookie 11731514351687301206 [ 8700.098953] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8700.430239] Lustre: server umount lustre-MDT0001 complete [ 8710.155221] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8715.496128] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8720.303360] Lustre: server umount MGS complete [ 8732.728088] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8733.106313] LustreError: 278497:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8733.133217] LustreError: 278510:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8733.176661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8738.283787] LustreError: 278498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8738.682106] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8747.495345] LustreError: 278498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8747.525184] LustreError: 278498:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8749.480876] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8754.784317] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8761.086242] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8765.158490] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8772.397043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8772.932575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8772.937948] Lustre: Skipped 1 previous similar message [ 8779.964516] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8785.464907] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8808.930795] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8808.933437] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8808.968668] Lustre: Skipped 1 previous similar message [ 8808.984763] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8808.991249] Lustre: Skipped 1 previous similar message [ 8813.248630] Lustre: server umount lustre-OST0000 complete [ 8819.173767] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8819.202502] Lustre: Skipped 1 previous similar message [ 8823.584842] Lustre: server umount lustre-MDT0000 complete [ 8824.293200] LustreError: 278493:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8824.311700] LustreError: 278493:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8827.695120] LustreError: 278478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775855390 with bad export cookie 11731514351687303124 [ 8827.701671] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8827.706113] LustreError: 278478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8828.033837] Lustre: server umount lustre-MDT0001 complete [ 8840.262924] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 8843.224482] Key type lgssc unregistered [ 8843.604518] LNet: 282088:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8843.616394] LNetError: 282088:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8844.647766] LNet: Removed LNI 192.168.202.127@tcp [ 8845.720572] Key type .llcrypt unregistered [ 8845.722684] Key type ._llcrypt unregistered [ 8892.195686] Key type ._llcrypt registered [ 8892.196924] Key type .llcrypt registered [ 8892.265118] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 8908.564988] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 8909.968469] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8909.983849] alg: No test for adler32 (adler32-zlib) [ 8911.152996] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 8911.416309] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 8913.039171] Key type lgssc registered [ 8914.609460] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8958.367689] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8958.395100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8959.800501] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8959.850897] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8959.948223] Lustre: lustre-MDT0000: new disk, initializing [ 8960.136289] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8960.182312] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8965.571224] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8977.624892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8977.789495] Lustre: 286017:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8977.888607] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8977.892450] Lustre: Skipped 1 previous similar message [ 8977.996481] Lustre: lustre-MDT0001: new disk, initializing [ 8978.153475] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8978.226128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8978.243556] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8983.119393] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 8991.043501] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8995.324188] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9002.468935] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9002.773600] Lustre: lustre-OST0000: new disk, initializing [ 9002.779285] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9002.906667] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9009.770718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9009.787645] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9009.907836] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9009.942334] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9018.439034] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9025.767868] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9038.449284] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9043.295419] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9048.409069] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9097.186742] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9097.188948] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9097.196996] LustreError: Skipped 1 previous similar message [ 9097.199778] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9097.215205] Lustre: Skipped 1 previous similar message [ 9100.269348] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9100.277168] Lustre: Skipped 2 previous similar messages [ 9101.889854] Lustre: server umount lustre-OST0000 complete [ 9106.406252] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9106.424852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9106.448852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9110.497358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9110.499657] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9115.618536] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9115.630249] Lustre: Skipped 2 previous similar messages [ 9120.223433] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9120.516688] Lustre: server umount lustre-MDT0000 complete [ 9124.384380] LustreError: 286009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775855687 with bad export cookie 6765654286517819213 [ 9124.392423] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9124.398325] LustreError: 286009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9124.826551] Lustre: server umount lustre-MDT0001 complete [ 9134.803213] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9140.179592] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9146.566877] Lustre: server umount MGS complete [ 9159.632384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9160.327465] LustreError: 291456:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9160.362890] LustreError: 291456:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9160.380189] LustreError: 291469:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9160.514351] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9165.792195] LustreError: 291457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9165.992964] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9170.914072] LustreError: 291456:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9176.040250] LustreError: 291457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9176.313455] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9176.864718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9183.004829] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9190.367923] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9195.321224] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9203.415545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9203.835632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9210.673553] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9217.202616] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9247.224048] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9247.240600] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9247.284109] LustreError: 293052:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9247.553712] Lustre: server umount lustre-OST0000 complete [ 9253.346921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9253.355683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9253.359020] Lustre: Skipped 1 previous similar message [ 9253.367561] Lustre: Skipped 1 previous similar message [ 9258.292620] Lustre: server umount lustre-MDT0000 complete [ 9258.475986] LustreError: 291457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9258.489827] LustreError: 291457:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9263.255125] LustreError: 293956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775855825 with bad export cookie 6765654286517821138 [ 9263.261000] LustreError: 293956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9263.264951] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9273.078789] Lustre: server umount lustre-MDT0001 complete [ 9286.417952] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [ 9289.922668] Key type lgssc unregistered [ 9290.213249] LNet: 295051:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9290.235762] LNetError: 295051:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9290.259185] LNet: Removed LNI 192.168.202.127@tcp [ 9291.136102] Key type .llcrypt unregistered [ 9291.140344] Key type ._llcrypt unregistered [ 9317.183699] Key type ._llcrypt registered [ 9317.187180] Key type .llcrypt registered [ 9317.267336] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [ 9331.470535] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 9333.788640] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9333.809867] alg: No test for adler32 (adler32-zlib) [ 9335.030573] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [ 9335.351261] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [ 9337.055179] Key type lgssc registered [ 9338.482732] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9370.399238] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [ 9391.632220] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9391.667368] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9393.047060] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9393.123107] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9393.316887] Lustre: lustre-MDT0000: new disk, initializing [ 9393.555721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9393.594674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9399.595408] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [ 9411.222075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9411.589752] Lustre: lustre-OST0000: new disk, initializing [ 9411.603244] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9411.606464] Lustre: Skipped 1 previous similar message [ 9411.692804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9418.446112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9418.453462] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9418.494556] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9419.358969] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10392.164138] Lustre: server umount lustre-MDT0000 complete [10396.074741] LustreError: 298241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775856958 with bad export cookie 7369798977255998674 [10396.091510] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10405.436555] Lustre: server umount lustre-OST0000 complete [10435.176848] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [10447.643839] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [10491.602816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10491.965586] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10492.016117] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10492.147275] Lustre: lustre-MDT0000: new disk, initializing [10492.342065] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10492.410185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10497.563737] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10509.975433] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10510.160347] Lustre: 303339:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10510.235305] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10510.240018] Lustre: Skipped 1 previous similar message [10510.327951] Lustre: lustre-MDT0001: new disk, initializing [10510.415530] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10510.462552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10510.473253] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10515.072680] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10525.483809] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10530.404897] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10538.171074] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10538.508750] Lustre: lustre-OST0000: new disk, initializing [10538.511637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10538.564975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10545.242478] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10547.835766] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10547.855624] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10548.012407] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10555.480783] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10561.261292] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10561.550688] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10567.909702] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10568.131925] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10573.793021] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10573.796704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10573.806740] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10573.834108] Lustre: Skipped 1 previous similar message [10577.097602] Lustre: server umount lustre-OST0000 complete [10581.986246] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10581.989528] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10582.013508] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10582.022372] Lustre: Skipped 1 previous similar message [10584.034140] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10584.034939] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10589.164394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10589.175379] Lustre: Skipped 2 previous similar messages [10594.273936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10594.285353] Lustre: Skipped 1 previous similar message [10595.295200] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10595.572689] Lustre: server umount lustre-MDT0000 complete [10599.570705] LustreError: 306274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775857162 with bad export cookie 7369798977258939493 [10599.581201] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10599.590488] LustreError: 306274:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10600.075386] Lustre: server umount lustre-MDT0001 complete [10603.932779] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10605.718507] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:39:26 (1775857166) [10612.571854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10612.854047] LustreError: 307250:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10612.868647] LustreError: 307250:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10612.948209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10618.339299] LustreError: 307251:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10618.448649] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10622.436194] LustreError: 307250:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10627.563853] LustreError: 307251:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10627.969294] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10628.508608] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10633.588362] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10641.198487] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10645.757321] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10654.806027] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10655.423076] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10662.116252] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10668.273787] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10676.907961] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10677.050170] Lustre: lustre-OST0001: new disk, initializing [10677.054279] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10677.144794] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10683.250094] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10684.550302] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10684.561143] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10684.628303] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10684.644556] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10685.667855] LustreError: 310177:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10686.693230] LustreError: 310177:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10686.702347] LustreError: 310177:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10688.806529] LustreError: 310177:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10688.828704] LustreError: 310177:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [10692.139888] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10692.324863] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10692.896917] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10692.910673] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [10697.253790] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9ba6c6175800.ost_server_uuid 50 [10699.011826] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9ba6c6175800.ost_server_uuid in FULL state after 0 sec [10701.176533] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10701.197833] LustreError: 310169:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10703.725747] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9ba6c6175800.ost_server_uuid 50 [10705.245714] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9ba6c6175800.ost_server_uuid in FULL state after 0 sec [10707.471850] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10720.740085] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10720.762776] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10726.834893] Lustre: server umount lustre-OST0001 complete [10729.125481] LustreError: 308845:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.27@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10734.560125] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10734.572839] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10734.581572] Lustre: Skipped 1 previous similar message [10738.581860] Lustre: server umount lustre-OST0000 complete [10746.336787] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10746.339139] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10746.339894] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10746.339899] Lustre: Skipped 8 previous similar messages [10746.363417] Lustre: Skipped 2 previous similar messages [10746.402594] LustreError: Skipped 1 previous similar message [10750.152853] Lustre: server umount lustre-MDT0000 complete [10751.459764] LustreError: 307251:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10751.477544] LustreError: 307251:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [10756.057411] LustreError: 309751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775857318 with bad export cookie 7369798977258940375 [10756.070716] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10756.084065] LustreError: 309751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10756.669827] Lustre: server umount lustre-MDT0001 complete [10773.345918] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [10776.769781] Key type lgssc unregistered [10777.165806] LNet: 312219:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10777.177475] LNetError: 312219:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10777.210744] LNet: Removed LNI 192.168.202.127@tcp [10778.606699] Key type .llcrypt unregistered [10778.608975] Key type ._llcrypt unregistered [10807.260165] Key type ._llcrypt registered [10807.267137] Key type .llcrypt registered [10807.475855] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [10808.878183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10808.922844] alg: No test for adler32 (adler32-zlib) [10810.072720] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [10810.430770] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [10812.175260] Key type lgssc registered [10813.580799] Lustre: Echo OBD driver; http://www.lustre.org/ [10821.868164] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10821.886897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10823.340741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10828.868666] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10833.891022] LustreError: 313622:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10833.907669] LustreError: 313622:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10836.428638] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10839.008780] LustreError: 313621:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10844.126239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10844.136055] LustreError: 313622:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10844.767278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10849.428350] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10860.350798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10860.838068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10867.665263] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10869.059224] LustreError: 315172:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10869.083881] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10873.862610] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10875.367283] LustreError: 315586:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10875.397665] LustreError: 315586:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10880.587320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10880.794555] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10886.141004] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10887.617855] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [10893.851647] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10900.196750] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10900.476957] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11004.384455] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11004.395581] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11004.414061] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11008.995039] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11009.003936] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11009.023092] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11009.028450] Lustre: Skipped 1 previous similar message [11010.202865] Lustre: server umount lustre-OST0001 complete [11012.069064] LustreError: 315175:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11012.075946] LustreError: 315175:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11017.187812] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11017.188556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11017.196121] Lustre: Skipped 1 previous similar message [11022.420445] Lustre: server umount lustre-OST0000 complete [11027.428551] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11027.448885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11027.460218] Lustre: Skipped 2 previous similar messages [11032.546594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11032.549280] Lustre: Skipped 1 previous similar message [11033.023102] Lustre: server umount lustre-MDT0000 complete [11037.001406] LustreError: 313603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775857599 with bad export cookie 11582831613147894406 [11037.006236] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11037.009440] LustreError: 313603:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11037.352519] Lustre: server umount lustre-MDT0001 complete [11048.118617] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [11051.561957] Key type lgssc unregistered [11051.857925] LNet: 318184:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11051.864843] LNetError: 318184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11051.877593] LNet: Removed LNI 192.168.202.127@tcp [11052.684716] Key type .llcrypt unregistered [11052.686569] Key type ._llcrypt unregistered [11101.133044] Key type ._llcrypt registered [11101.137523] Key type .llcrypt registered [11101.353098] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [11102.592650] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11102.619940] alg: No test for adler32 (adler32-zlib) [11103.869700] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [11104.158326] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [11105.911205] Key type lgssc registered [11107.097580] Lustre: Echo OBD driver; http://www.lustre.org/ [11116.704740] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11116.727854] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11118.213384] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11122.409519] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11127.794351] LustreError: 320465:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11127.817712] LustreError: 320465:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11131.405891] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11131.854857] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11136.300402] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11139.568784] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11146.820950] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11147.111557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11153.384687] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11156.326334] LustreError: 321919:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11160.960358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11161.143971] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11163.214516] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11163.221534] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11167.299498] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11175.490708] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11182.385043] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11198.433720] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11198.440698] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11198.454774] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11198.948854] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11198.954927] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11198.958642] Lustre: Skipped 1 previous similar message [11198.971376] Lustre: Skipped 2 previous similar messages [11204.066023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11204.071790] Lustre: Skipped 3 previous similar messages [11204.581831] Lustre: server umount lustre-MDT0000 complete [11208.161972] LustreError: 320445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775857770 with bad export cookie 1247287152867276485 [11208.165765] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11208.177335] LustreError: 320445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11208.511406] Lustre: server umount lustre-MDT0001 complete [11222.260770] Lustre: server umount lustre-OST0000 complete [11224.544138] Lustre: 319729:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775857771/real 1775857771] req@ffff8c3388b51180 x1862121728045824/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775857787 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11224.567189] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11224.581940] Lustre: Skipped 1 previous similar message [11225.419433] Lustre: server umount lustre-OST0001 complete [11242.784868] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [11253.349667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11253.808088] LustreError: 325652:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11253.900854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11259.023251] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11259.360906] LustreError: 325653:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11263.478441] LustreError: 325652:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11273.733754] LustreError: 325653:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11273.781235] LustreError: 325653:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11276.503315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11277.272170] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11284.242665] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11288.798795] Lustre: 326759:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11297.002378] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11297.805490] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11298.805891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11298.821570] LustreError: 327174:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11305.085388] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11314.808457] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11316.933743] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11322.273605] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11330.317519] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11333.872408] Lustre: 328606:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11343.844772] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11343.856886] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11343.874363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11349.393351] LustreError: 328709:0:(ldlm_resource.c:1170:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (ffff8c3399a87700) refcount nonzero (1) after lock cleanup; forcing cleanup. [11349.630692] Lustre: server umount lustre-MDT0000 complete [11351.011941] LustreError: 325653:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11351.031977] LustreError: 325653:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [11353.934253] LustreError: 327138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775857916 with bad export cookie 1247287152867278186 [11353.939419] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11353.944477] LustreError: 327138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11354.290955] Lustre: server umount lustre-MDT0001 complete [11369.401567] Lustre: server umount lustre-OST0000 complete [11371.488157] Lustre: 319726:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775857918/real 1775857918] req@ffff8c348d9bea00 x1862121728104192/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775857934 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11371.551708] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11371.581591] Lustre: Skipped 3 previous similar messages [11373.535433] Lustre: 319728:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1775857920/real 1775857920] req@ffff8c3485eaf100 x1862121728104448/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1775857936 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11374.362419] Lustre: server umount lustre-OST0001 complete [11378.369263] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11380.637323] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11382.782860] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:52:23 (1775857943) [11406.627858] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [11416.919533] loop0: detected capacity change from 0 to 1099511627776 [11417.627072] SGI XFS with ACLs, security attributes, quota, no debug enabled [11417.643442] *** VALIDATE xfs *** [11417.674693] XFS (loop0): Mounting V5 Filesystem [11417.725806] XFS (loop0): Ending clean mount [11419.009239] loop1: detected capacity change from 0 to 18691697672192 [11517.122803] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [11528.818808] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [11569.887731] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11570.169897] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11570.221300] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11570.366411] Lustre: lustre-MDT0000: new disk, initializing [11570.513550] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11570.518868] Lustre: Skipped 1 previous similar message [11570.536354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11575.316180] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11591.249413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11591.619561] Lustre: 333705:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11591.780983] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11591.793569] Lustre: Skipped 1 previous similar message [11591.994809] Lustre: lustre-MDT0001: new disk, initializing [11592.222541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11592.253138] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11598.763666] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11607.775846] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11613.098659] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11621.543987] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11621.937754] Lustre: lustre-OST0000: new disk, initializing [11621.945563] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11623.176368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11623.189269] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11623.401141] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11629.993696] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11639.015935] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11644.083393] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11644.391733] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11650.301691] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11650.577857] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11654.124986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11654.125877] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11654.143481] LustreError: Skipped 1 previous similar message [11654.158597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11654.161408] Lustre: Skipped 4 previous similar messages [11659.144962] Lustre: server umount lustre-OST0000 complete [11663.841320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11663.848736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11669.706400] Lustre: server umount lustre-MDT0000 complete [11673.575746] LustreError: 333717:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11673.589568] LustreError: 333717:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11673.922554] LustreError: 333697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775858236 with bad export cookie 1247287152867279740 [11673.928977] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11673.936411] LustreError: 333697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11674.382527] Lustre: server umount lustre-MDT0001 complete [11683.363603] XFS (loop0): Unmounting Filesystem [11688.930416] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:57:29 (1775858249) [11696.285199] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11697.055662] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11697.060939] Lustre: Skipped 2 previous similar messages [11701.860981] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11709.669600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11713.861896] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11721.729754] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11726.313275] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11735.633683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11744.184703] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11750.688882] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11767.778219] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11767.783405] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11767.797709] Lustre: Skipped 3 previous similar messages [11767.801669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11767.806115] Lustre: Skipped 4 previous similar messages [11771.936794] Lustre: server umount lustre-OST0000 complete [11776.992809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11777.007524] LustreError: Skipped 1 previous similar message [11791.327184] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11791.581604] Lustre: server umount lustre-MDT0000 complete [11796.280735] LustreError: 337841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775858358 with bad export cookie 1247287152867280622 [11796.299039] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11796.311764] LustreError: 337841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11796.753451] Lustre: server umount lustre-MDT0001 complete [11808.284668] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [11812.582919] Key type lgssc unregistered [11813.080092] LNet: 341144:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11813.090285] LNetError: 341144:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11813.108540] LNet: Removed LNI 192.168.202.127@tcp [11814.151168] Key type .llcrypt unregistered [11814.153350] Key type ._llcrypt unregistered [11845.120644] Key type ._llcrypt registered [11845.122637] Key type .llcrypt registered [11845.275962] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [11846.998193] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11847.182958] alg: No test for adler32 (adler32-zlib) [11848.352555] Lustre: Lustre: Build Version: 2.17.52_1_g064474b [11848.584423] LNet: Added LNI 192.168.202.127@tcp [8/256/0/180] [11850.335228] Key type lgssc registered [11851.172978] Lustre: Echo OBD driver; http://www.lustre.org/ [11861.462201] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11861.504771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11863.129288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11869.105321] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11873.261020] LustreError: 342695:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11873.291906] LustreError: 342695:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11878.377318] LustreError: 342696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11880.623441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11881.284157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11887.064815] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11894.956786] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11900.578752] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11911.158940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11911.763774] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11919.375338] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11927.866325] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11942.880625] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11942.908730] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11942.937621] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11943.905172] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11943.907444] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11949.038437] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11949.052658] Lustre: Skipped 3 previous similar messages [11954.151885] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11954.160765] Lustre: Skipped 2 previous similar messages [11955.168474] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11955.719726] Lustre: server umount lustre-MDT0001 complete [11959.273446] LustreError: 342692:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11959.311984] LustreError: 342692:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11964.390682] LustreError: 342695:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11964.438155] LustreError: 342695:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11969.528495] LustreError: 342690:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11969.567282] LustreError: 342690:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11971.895215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11972.113228] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11972.131676] Lustre: Found index 1 for lustre-MDT0001, updating log [11972.186850] Lustre: 342684:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11972.567952] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11976.869378] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [11977.725680] LustreError: 345861:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11977.727310] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11977.735340] LustreError: 345861:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11977.735630] LustreError: 342687:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11977.772170] LustreError: 345869:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.202.127@tcp: cfg command failed: rc = -17 [11977.784160] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11977.784160] [12284.907877] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12284.915765] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12284.931488] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12284.957992] Lustre: Skipped 2 previous similar messages [12289.556843] Lustre: server umount lustre-MDT0001 complete [12290.024301] LustreError: 342692:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12290.045459] LustreError: 342692:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12303.258267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12303.521253] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12303.538143] Lustre: Found index 1 for lustre-MDT0001, updating log [12303.610357] Lustre: 342683:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12304.116538] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12309.492726] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12309.525233] LustreError: 353513:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12309.532365] LustreError: 353513:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12309.544913] LustreError: 342687:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12309.555059] LustreError: 342687:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12309.628986] LustreError: 353518:0:(obd_config.c:760:class_setup()) Device 8 already setup (type osp) [12309.634865] LustreError: 353518:0:(obd_config.c:864:class_detach()) OBD device 8 still set up [12309.640874] LustreError: 353518:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.202.127@tcp: cfg command failed: rc = -17 [12309.646980] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.127@tcp [12309.646980] [12310.793340] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [12620.502148] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.27@tcp (stopping) [12620.513471] Lustre: Skipped 1 previous similar message [12621.793306] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12621.793674] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12621.819500] Lustre: Skipped 1 previous similar message [12623.553828] Lustre: server umount lustre-MDT0001 complete [12625.567728] LustreError: 342691:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.27@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12625.590700] LustreError: 342691:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [12635.571750] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12635.763826] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12635.767381] Lustre: Found index 1 for lustre-MDT0001, updating log [12635.808329] Lustre: 342683:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12636.313992] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12641.706716] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [12641.767631] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12643.319535] LustreError: 361345:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12643.332883] LustreError: 361345:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12643.341509] LustreError: 342687:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12643.356760] LustreError: 342687:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12643.424909] LustreError: 361350:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12643.436918] LustreError: 361350:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.202.127@tcp: cfg command failed: rc = -114 [12643.444261] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12643.444261] [12962.272722] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12962.277244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12962.284403] Lustre: Skipped 3 previous similar messages [12962.308112] Lustre: Skipped 4 previous similar messages [12968.431485] Lustre: server umount lustre-MDT0000 complete [12971.971493] LustreError: 342676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775859534 with bad export cookie 9051814239926907552 [12971.979586] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12971.986193] LustreError: 342676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12972.513880] LustreError: 350062:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12972.514691] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12972.549117] LustreError: 350062:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [12986.336518] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12986.670110] Lustre: server umount lustre-MDT0001 complete [12991.388782] Lustre: server umount lustre-OST0000 complete [13002.949194] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [13014.491995] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [13058.241433] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 18:20:19 (1775859619) [13065.470423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13065.768126] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13065.820360] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13065.916893] Lustre: lustre-MDT0000: new disk, initializing [13066.075513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13066.101060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13070.599144] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [13081.393189] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13081.553306] Lustre: 371779:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13081.633575] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13081.638935] Lustre: Skipped 1 previous similar message [13081.711585] Lustre: lustre-MDT0001: new disk, initializing [13081.806465] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13081.847623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13081.861109] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13087.234254] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [13094.874530] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13099.282051] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13106.564133] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13106.796932] Lustre: lustre-OST0000: new disk, initializing [13106.801056] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13106.911616] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13113.421524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13113.434903] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13113.590266] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13113.739856] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_default_debug -1 all [13121.915721] Lustre: DEBUG MARKER: oleg227-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13130.731175] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13130.742811] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13130.763461] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13130.774759] Lustre: Skipped 6 previous similar messages [13131.550219] Lustre: server umount lustre-MDT0000 complete [13134.308044] LustreError: 371791:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13134.345301] LustreError: 371791:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [13135.394414] LustreError: 371774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1775859698 with bad export cookie 9051814239926909498 [13135.403160] LustreError: MGC192.168.202.127@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13135.414182] LustreError: 371774:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13135.414368] LustreError: 372571:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13135.414502] LustreError: 372571:0:(client.c:1380:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@000000002c57b173 x1862122509569792/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [13135.809857] Lustre: server umount lustre-MDT0001 complete [13150.256816] Lustre: server umount lustre-OST0000 complete [13196.913708] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing set_hostid [13208.213743] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing load_modules_local [13254.998083] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13024 sec ========= 18:23:35 (1775859815) [13257.453828] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:23:37 (1775859817) === [13261.407625] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:23:42 (1775859822) === [13312.400139] Lustre: DEBUG MARKER: oleg227-server.virtnet: executing unload_modules_local [13316.213563] Key type lgssc unregistered [13316.628966] LNet: 379731:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13316.639359] LNetError: 379731:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13316.659849] LNet: Removed LNI 192.168.202.127@tcp [13317.765508] Key type .llcrypt unregistered [13317.767685] Key type ._llcrypt unregistered