[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 538235400 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.003255] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005016] kvm-guest: setup PV IPIs [ 0.008692] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010015] pid_max: default: 32768 minimum: 301 [ 0.011148] LSM: Security Framework initializing [ 0.012061] Yama: becoming mindful. [ 0.013038] SELinux: Initializing. [ 0.014081] *** VALIDATE selinux *** [ 0.022840] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027509] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028159] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029108] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030116] *** VALIDATE tmpfs *** [ 0.032090] *** VALIDATE proc *** [ 0.033247] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036200] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037156] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039031] Spectre V2 : User space: Vulnerable [ 0.040010] Speculative Store Bypass: Vulnerable [ 0.042870] debug: unmapping init [mem 0xffffffff91c59000-0xffffffff91c60fff] [ 0.044196] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045718] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046023] ... version: 2 [ 0.047013] ... bit width: 48 [ 0.048014] ... generic registers: 4 [ 0.049016] ... value mask: 0000ffffffffffff [ 0.050013] ... max period: 00007fffffffffff [ 0.051011] ... fixed-purpose events: 3 [ 0.051998] ... event mask: 000000070000000f [ 0.052296] rcu: Hierarchical SRCU implementation. [ 0.054641] smp: Bringing up secondary CPUs ... [ 0.055662] x86: Booting SMP configuration: [ 0.056028] .... node #0, CPUs: #1 #2 #3 [ 0.059291] smp: Brought up 1 node, 4 CPUs [ 0.061016] smpboot: Max logical packages: 1 [ 0.062017] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.136073] node 0 deferred pages initialised in 72ms [ 0.140231] devtmpfs: initialized [ 0.142428] x86/mm: Memory block size: 128MB [ 0.145891] gcov: version magic: 0x41383552 [ 0.149443] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.154087] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.156439] pinctrl core: initialized pinctrl subsystem [ 0.158466] [ 0.158874] ************************************************************* [ 0.162139] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.165013] ** ** [ 0.168015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.171019] ** ** [ 0.173025] ** This means that this kernel is built to expose internal ** [ 0.176017] ** IOMMU data structures, which may compromise security on ** [ 0.179017] ** your system. ** [ 0.181013] ** ** [ 0.184017] ** If you see this message and you are not debugging the ** [ 0.187018] ** kernel, report this immediately to your vendor! ** [ 0.189012] ** ** [ 0.191028] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.194016] ************************************************************* [ 0.197704] NET: Registered protocol family 16 [ 0.199477] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.202070] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.205064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.208501] cpuidle: using governor menu [ 0.209839] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.214530] PCI: Using configuration type 1 for base access [ 0.216148] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.226152] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.229065] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.234046] cryptd: max_cpu_qlen set to 1000 [ 0.237334] ACPI: Added _OSI(Module Device) [ 0.239022] ACPI: Added _OSI(Processor Device) [ 0.241024] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.242019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.247674] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.255261] ACPI: Interpreter enabled [ 0.256109] ACPI: PM: (supports S0 S3 S4 S5) [ 0.258023] ACPI: Using IOAPIC for interrupt routing [ 0.260166] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.266289] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.275366] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.276039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.279025] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.281068] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.285184] acpiphp: Slot [2] registered [ 0.286136] acpiphp: Slot [5] registered [ 0.287212] acpiphp: Slot [6] registered [ 0.288098] acpiphp: Slot [7] registered [ 0.290160] acpiphp: Slot [8] registered [ 0.291208] acpiphp: Slot [9] registered [ 0.293306] acpiphp: Slot [10] registered [ 0.295170] acpiphp: Slot [3] registered [ 0.296185] acpiphp: Slot [4] registered [ 0.298129] acpiphp: Slot [11] registered [ 0.300216] acpiphp: Slot [12] registered [ 0.302205] acpiphp: Slot [13] registered [ 0.303124] acpiphp: Slot [14] registered [ 0.305130] acpiphp: Slot [15] registered [ 0.306170] acpiphp: Slot [16] registered [ 0.308127] acpiphp: Slot [17] registered [ 0.309145] acpiphp: Slot [18] registered [ 0.311119] acpiphp: Slot [19] registered [ 0.313478] acpiphp: Slot [20] registered [ 0.315242] acpiphp: Slot [21] registered [ 0.317146] acpiphp: Slot [22] registered [ 0.319166] acpiphp: Slot [23] registered [ 0.320131] acpiphp: Slot [24] registered [ 0.322227] acpiphp: Slot [25] registered [ 0.324164] acpiphp: Slot [26] registered [ 0.326155] acpiphp: Slot [27] registered [ 0.327084] acpiphp: Slot [28] registered [ 0.329132] acpiphp: Slot [29] registered [ 0.331180] acpiphp: Slot [30] registered [ 0.332141] acpiphp: Slot [31] registered [ 0.334071] PCI host bridge to bus 0000:00 [ 0.335022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.338026] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.342028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.344028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.346025] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.349028] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.351255] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.355642] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.359684] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.370030] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.377123] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.380032] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.382033] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.385044] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.388332] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.390955] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.394050] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.396954] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.402025] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.415035] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.421959] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.427396] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.437025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.444025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.462029] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.473532] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.479019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.484016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.498017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.508830] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.516026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.527028] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.546033] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.558044] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.564023] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.575030] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.593032] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.616113] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.623040] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.632028] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.652027] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.663662] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.670028] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.677031] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.698029] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.718255] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.721508] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.724610] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.727577] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.730428] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.735434] iommu: Default domain type: Passthrough [ 0.738069] SCSI subsystem initialized [ 0.740426] ACPI: bus type USB registered [ 0.742234] usbcore: registered new interface driver usbfs [ 0.745119] usbcore: registered new interface driver hub [ 0.747115] usbcore: registered new device driver usb [ 0.749231] pps_core: LinuxPPS API ver. 1 registered [ 0.751013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.754097] PTP clock support registered [ 0.756228] EDAC MC: Ver: 3.0.0 [ 0.759132] PCI: Using ACPI for IRQ routing [ 0.761018] NetLabel: Initializing [ 0.763014] NetLabel: domain hash size = 128 [ 0.764012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.766121] NetLabel: unlabeled traffic allowed by default [ 0.768173] vgaarb: loaded [ 0.770351] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.772024] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.781000] clocksource: Switched to clocksource kvm-clock [ 0.899057] VFS: Disk quotas dquot_6.6.0 [ 0.900380] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.903286] *** VALIDATE ramfs *** [ 0.904722] *** VALIDATE hugetlbfs *** [ 0.907552] pnp: PnP ACPI init [ 0.910206] pnp: PnP ACPI: found 6 devices [ 0.927148] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.930553] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.932885] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.935171] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.937339] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.939477] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.942421] NET: Registered protocol family 2 [ 0.945031] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.950250] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.953794] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.958794] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.962902] TCP: Hash tables configured (established 65536 bind 65536) [ 0.966525] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.970319] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.973450] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.976920] NET: Registered protocol family 1 [ 0.979906] RPC: Registered named UNIX socket transport module. [ 0.982158] RPC: Registered udp transport module. [ 0.984455] RPC: Registered tcp transport module. [ 0.986544] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.989387] NET: Registered protocol family 44 [ 0.991310] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.993742] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.996239] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.998855] PCI: CLS 0 bytes, default 64 [ 1.000881] Unpacking initramfs... [ 2.451382] debug: unmapping init [mem 0xffff98dbbcc54000-0xffff98dbbffbffff] [ 2.454678] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.456513] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.459319] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.994385] Initialise system trusted keyrings [ 2.996352] Key type blacklist registered [ 2.999180] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.008639] zbud: loaded [ 3.013423] *** VALIDATE nfs *** [ 3.014353] *** VALIDATE nfs4 *** [ 3.015803] pstore: using deflate compression [ 3.019179] Platform Keyring initialized [ 3.152958] NET: Registered protocol family 38 [ 3.154485] Key type asymmetric registered [ 3.156137] Asymmetric key parser 'x509' registered [ 3.157712] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.159905] io scheduler mq-deadline registered [ 3.161167] io scheduler kyber registered [ 3.162579] io scheduler bfq registered [ 3.167553] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.169604] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.171655] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.173629] ACPI: Power Button [PWRF] [ 3.178486] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.183107] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.191266] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.195785] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.206300] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.231721] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.259408] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.266149] Non-volatile memory driver v1.3 [ 3.267559] Linux agpgart interface v0.103 [ 3.291597] virtio_blk virtio1: [vda] 134872 512-byte logical blocks (69.1 MB/65.9 MiB) [ 3.295019] vda: detected capacity change from 0 to 69054464 [ 3.306551] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.308863] vdb: detected capacity change from 0 to 1073741824 [ 3.325643] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.327809] vdc: detected capacity change from 0 to 2621440000 [ 3.342804] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.345719] vdd: detected capacity change from 0 to 2621440000 [ 3.358068] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.360592] vde: detected capacity change from 0 to 4294967296 [ 3.375242] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.378457] vdf: detected capacity change from 0 to 4294967296 [ 3.386822] libphy: Fixed MDIO Bus: probed [ 3.395696] usbcore: registered new interface driver usbserial_generic [ 3.398530] usbserial: USB Serial support registered for generic [ 3.401201] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.406534] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.408647] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.411238] mousedev: PS/2 mouse device common for all mice [ 3.414381] rtc_cmos 00:05: RTC can wake from S4 [ 3.417597] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.421606] rtc_cmos 00:05: registered as rtc0 [ 3.423094] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.424323] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.429875] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.430914] intel_pstate: CPU model not supported [ 3.437710] hid: raw HID events driver (C) Jiri Kosina [ 3.440170] usbcore: registered new interface driver usbhid [ 3.442283] usbhid: USB HID core driver [ 3.443774] drop_monitor: Initializing network drop monitor service [ 3.446432] Initializing XFRM netlink socket [ 3.448457] NET: Registered protocol family 10 [ 3.451689] Segment Routing with IPv6 [ 3.453318] NET: Registered protocol family 17 [ 3.456055] mpls_gso: MPLS GSO support [ 3.462507] RAS: Correctable Errors collector initialized. [ 3.464798] AVX version of gcm_enc/dec engaged. [ 3.466417] AES CTR mode by8 optimization enabled [ 3.557988] sched_clock: Marking stable (3557967467, 0)->(4494467366, -936499899) [ 3.562152] registered taskstats version 1 [ 3.564721] Loading compiled-in X.509 certificates [ 3.566669] zswap: loaded using pool lzo/zbud [ 3.593404] Key type big_key registered [ 3.606132] Key type encrypted registered [ 3.607675] ima: No TPM chip found, activating TPM-bypass! [ 3.609638] ima: Allocated hash algorithm: sha1 [ 3.611254] ima: No architecture policies found [ 3.613094] evm: Initialising EVM extended attributes: [ 3.614541] evm: security.selinux [ 3.615592] evm: security.ima [ 3.616409] evm: security.capability [ 3.617819] evm: HMAC attrs: 0x1 [ 3.621837] rtc_cmos 00:05: setting system clock to 2026-06-01 17:09:50 UTC (1780333790) [ 3.646426] debug: unmapping init [mem 0xffffffff92c03000-0xffffffff92dfffff] [ 3.652164] debug: unmapping init [mem 0xffffffff91982000-0xffffffff91c58fff] [ 3.663078] Write protecting the kernel read-only data: 28672k [ 3.666189] debug: unmapping init [mem 0xffffffff90003000-0xffffffff901fffff] [ 3.669018] debug: unmapping init [mem 0xffffffff90914000-0xffffffff909fffff] [ 3.723630] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.730463] systemd[1]: Detected virtualization kvm. [ 3.732197] systemd[1]: Detected architecture x86-64. [ 3.734408] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.765137] systemd[1]: No hostname configured. [ 3.766649] systemd[1]: Set hostname to . [ 3.768899] random: systemd: uninitialized urandom read (16 bytes read) [ 3.771128] systemd[1]: Initializing machine ID from random generator. [ 3.807573] random: ln: uninitialized urandom read (6 bytes read) [ 3.892543] random: systemd: uninitialized urandom read (16 bytes read) [ 3.895114] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.899559] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.903849] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Paths. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Journal Service. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.458985] device-mapper: uevent: version 1.0.3 [ 4.461325] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook..[ 5.128987] virtio_net virtio0 ens2: renamed from eth0 . [ 5.153141] random: fast init done [ 5.253618] scsi host0: ata_piix [ 5.278989] scsi host1: ata_piix [ 5.280615] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.282989] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.618765] dracut-initqueue[593]: RTNETLINK answers: File exists [ 10.132840] random: crng init done [ 10.135026] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.466443] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 14.576434] printk: systemd: 25 output lines suppressed due to ratelimiting [ 15.260336] SELinux: Disabled at runtime. [ 15.391473] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 15.411630] systemd[1]: Detected virtualization kvm. [ 15.414749] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.147234] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.158113] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.180321] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.198485] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.208373] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.229915] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.251543] systemd[1]: proc-sys-fs-binfmt_misc.automount: Refusing to start, unit to trigger not loaded. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 17.615745] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. Starting Flush Journal to Persistent Storage... Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 19.213601] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.494607] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 20.561948] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 21.182832] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 21.249727] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.005418] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 27.846298] NFS: Registering the id_resolver key type [ 27.851824] Key type id_resolver registered [ 27.857075] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (10s / no limit) [ **] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg231-server login: [ 40.948133] hrtimer: interrupt took 2191161 ns [ 96.600078] libcfs: loading out-of-tree module taints kernel. [ 96.650802] Key type ._llcrypt registered [ 96.658344] Key type .llcrypt registered [ 96.815136] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 118.856583] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 120.296838] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 120.305681] alg: No test for adler32 (adler32-zlib) [ 122.044620] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 122.861658] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 124.583199] Key type lgssc registered [ 127.144577] Lustre: Echo OBD driver; http://www.lustre.org/ [ 148.347275] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 193.798112] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 208.381449] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 208.421149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 209.701659] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 209.730766] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 209.821603] Lustre: lustre-MDT0000: new disk, initializing [ 209.914435] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 209.939304] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 214.704756] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 230.250414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 230.376517] Lustre: 6518:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 230.410907] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 230.415526] Lustre: Skipped 1 previous similar message [ 230.501127] Lustre: lustre-MDT0001: new disk, initializing [ 230.604033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 230.625458] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 230.636547] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 236.467941] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 241.897622] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 254.626227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 254.941875] Lustre: lustre-OST0000: new disk, initializing [ 254.947523] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 254.960186] Lustre: 8424:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 255.064585] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 261.682201] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 261.711929] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 261.788748] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 262.439966] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 279.265321] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 279.392333] Lustre: lustre-OST0001: new disk, initializing [ 279.394459] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 279.404577] Lustre: 9479:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 279.483928] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 285.247081] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 285.277586] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 285.341087] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 286.240619] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 300.608438] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 306.752161] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 315.628897] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing check_logdir /tmp/testlogs/ [ 322.456085] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing yml_node [ 327.224281] Lustre: DEBUG MARKER: Client: 2.17.53.35 [ 330.285601] Lustre: DEBUG MARKER: MDS: 2.17.53.35 [ 333.482953] Lustre: DEBUG MARKER: OSS: 2.17.53.35 [ 335.545809] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Jun 1 13:15:20 EDT 2026 [ 355.812330] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 357.645598] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 374.251457] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 374.268491] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 374.298761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 377.315701] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 377.319596] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 377.332421] Lustre: Skipped 2 previous similar messages [ 377.344318] Lustre: Skipped 3 previous similar messages [ 380.028395] Lustre: server umount lustre-MDT0000 complete [ 384.442483] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334171 with bad export cookie 17518319614054190618 [ 384.443891] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 384.458863] LustreError: 6509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 387.551859] LustreError: 6525:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 387.551959] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 387.556681] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 387.576853] LustreError: 6525:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 387.592590] Lustre: Skipped 1 previous similar message [ 389.601568] LustreError: 6523:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 389.611891] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 389.647285] Lustre: Skipped 2 previous similar messages [ 390.940892] Lustre: server umount lustre-MDT0001 complete [ 395.410922] Lustre: server umount lustre-OST0000 complete [ 400.631046] Lustre: server umount lustre-OST0001 complete [ 408.636077] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 419.589671] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 461.944235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 462.384554] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 462.451424] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 462.604633] Lustre: lustre-MDT0000: new disk, initializing [ 462.688944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 462.707456] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 468.274166] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 483.506209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 483.621541] Lustre: 14998:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 483.693269] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 483.703675] Lustre: Skipped 1 previous similar message [ 483.789163] Lustre: lustre-MDT0001: new disk, initializing [ 483.875911] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 483.920922] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 483.935933] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 488.820681] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 499.220319] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 503.628491] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 511.573345] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 511.930282] Lustre: lustre-OST0000: new disk, initializing [ 511.936898] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 511.942577] Lustre: 16746:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 512.050648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 513.399883] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 513.416891] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 513.586727] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 518.759953] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 527.714643] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 532.855586] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 533.172600] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 537.844621] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 538.001787] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 544.230023] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 544.235233] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 544.241896] LustreError: Skipped 1 previous similar message [ 544.258850] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 544.261793] Lustre: Skipped 1 previous similar message [ 544.271344] Lustre: Skipped 1 previous similar message [ 546.208178] Lustre: server umount lustre-OST0000 complete [ 550.368548] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 550.381042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 554.464903] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 554.472642] Lustre: Skipped 2 previous similar messages [ 564.191400] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 564.469774] Lustre: server umount lustre-MDT0000 complete [ 564.715752] LustreError: 15010:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 564.735278] LustreError: 15010:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 568.396165] LustreError: 16745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334355 with bad export cookie 17518319614054192193 [ 568.411181] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 568.426339] LustreError: 16745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 568.717565] Lustre: server umount lustre-MDT0001 complete [ 574.704779] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 13:19:19 (1780334359) [ 583.320396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 583.683025] LustreError: 18865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 583.696888] LustreError: 18865:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 583.746118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 588.790738] LustreError: 18866:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 588.827521] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 600.006057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 600.032260] LustreError: 18865:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.046559] LustreError: 18865:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 606.115752] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 612.185936] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 616.578809] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 623.834098] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 630.859685] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 637.689312] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 641.495921] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 651.745321] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 651.755981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 651.775334] Lustre: Skipped 1 previous similar message [ 651.787486] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 651.794379] Lustre: Skipped 2 previous similar messages [ 656.307931] Lustre: server umount lustre-MDT0000 complete [ 659.949954] LustreError: 18860:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 659.973293] LustreError: 18860:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 660.198088] LustreError: 18845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334447 with bad export cookie 17518319614054193117 [ 660.202799] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 660.220726] LustreError: 18845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 660.612719] Lustre: server umount lustre-MDT0001 complete [ 675.343274] Lustre: server umount lustre-OST0000 complete [ 703.407921] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 715.259132] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 715.771115] LustreError: 22916:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 715.874526] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 715.878492] Lustre: Skipped 2 previous similar messages [ 721.520198] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 732.755735] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 736.610950] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 740.262704] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 748.514971] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 757.235907] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 768.526941] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 768.719132] Lustre: lustre-OST0001: new disk, initializing [ 768.722262] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 768.728906] Lustre: 25183:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 770.795150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 770.826978] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 771.022518] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 776.439996] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 787.169702] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 791.258556] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 806.903864] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 806.954735] Lustre: Skipped 4 previous similar messages [ 806.971135] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 806.973015] Lustre: Skipped 3 previous similar messages [ 808.449974] Lustre: server umount lustre-MDT0000 complete [ 812.001052] LustreError: 22916:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 812.043668] LustreError: 22916:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 813.447159] LustreError: 25182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334600 with bad export cookie 17518319614054193964 [ 813.448677] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 813.484460] LustreError: 25182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 814.118940] Lustre: server umount lustre-MDT0001 complete [ 829.747754] Lustre: server umount lustre-OST0000 complete [ 830.175531] Lustre: 3666:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780334601/real 1780334601] req@ffff98dc32c25c00 x1866815409230720/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780334617 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 834.018439] Lustre: 3665:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780334604/real 1780334604] req@ffff98db05763480 x1866815409230976/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780334620 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 834.495984] Lustre: server umount lustre-OST0001 complete [ 852.944825] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 865.252501] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 865.922961] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 865.928320] Lustre: Skipped 3 previous similar messages [ 871.580501] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 883.538649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 890.624373] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 894.454684] Lustre: 29282:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 902.139989] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 909.677387] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 920.955058] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 928.465856] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 937.833197] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 942.059618] Lustre: 31130:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 950.753106] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 950.774056] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 950.794525] Lustre: Skipped 2 previous similar messages [ 950.815799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 950.824229] Lustre: Skipped 2 previous similar messages [ 955.644577] Lustre: server umount lustre-MDT0000 complete [ 957.422643] LustreError: 28162:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 957.479629] LustreError: 28162:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 960.460579] LustreError: 28148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334747 with bad export cookie 17518319614054195259 [ 960.463665] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 960.474852] LustreError: 28148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 960.765277] Lustre: server umount lustre-MDT0001 complete [ 975.740161] Lustre: server umount lustre-OST0000 complete [ 978.913761] Lustre: 3665:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780334749/real 1780334749] req@ffff98dc3dcdc000 x1866815409285888/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780334765 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 980.557314] Lustre: server umount lustre-OST0001 complete [ 992.389305] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 13:26:16 (1780334776) [ 1026.598198] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1038.553899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1044.990933] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1055.574227] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1061.284711] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1064.862384] Lustre: 35037:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1072.254255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1079.570777] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1090.292815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1098.964924] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1108.222255] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1112.805571] Lustre: 36878:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1121.766846] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1121.771626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1121.782217] Lustre: Skipped 7 previous similar messages [ 1121.797659] Lustre: Skipped 7 previous similar messages [ 1126.534577] Lustre: server umount lustre-MDT0000 complete [ 1131.399823] LustreError: 34654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780334918 with bad export cookie 17518319614054196365 [ 1131.402975] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1131.416026] LustreError: 34654:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1131.807043] Lustre: server umount lustre-MDT0001 complete [ 1147.359492] Lustre: 3664:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780334918/real 1780334918] req@ffff98dc0e1bce00 x1866815409339648/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780334934 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1147.540873] Lustre: server umount lustre-OST0000 complete [ 1151.455230] Lustre: 3667:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780334922/real 1780334922] req@ffff98dc3e637480 x1866815409339904/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780334938 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1153.011299] Lustre: server umount lustre-OST0001 complete [ 1164.215065] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 13:29:08 (1780334948) [ 1186.020125] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1198.039700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1198.664859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1198.671756] Lustre: Skipped 7 previous similar messages [ 1203.978663] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1213.929289] LustreError: 39006:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1213.955898] LustreError: 39006:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 16 previous similar messages [ 1214.660509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1219.995459] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1223.520923] Lustre: 40121:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1232.143384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1239.574335] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1249.735779] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1258.575670] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1270.221528] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1275.048185] Lustre: 41965:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1276.235657] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1276.244540] Lustre: Skipped 1 previous similar message [ 1280.996952] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1281.017960] Lustre: Skipped 4 previous similar messages [ 1286.166988] Lustre: server umount lustre-MDT0000 complete [ 1291.330424] LustreError: 40123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780335078 with bad export cookie 17518319614054197471 [ 1291.338661] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1291.339153] LustreError: 40123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1297.926584] Lustre: server umount lustre-MDT0001 complete [ 1307.948869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1316.832809] LustreError: 3663:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff98db08a7f100 x1866815409398144/t0(0) o250->MGC192.168.202.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1323.222123] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1326.449601] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1333.024496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1336.497461] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1336.505801] Lustre: Skipped 1 previous similar message [ 1338.720936] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1345.624912] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1348.163376] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1352.713347] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1354.635418] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1367.690256] Lustre: server umount lustre-MDT0000 complete [ 1372.111528] LustreError: 40123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780335158 with bad export cookie 17518319614054198598 [ 1372.120580] LustreError: 40123:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1372.125174] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1372.591708] Lustre: server umount lustre-MDT0001 complete [ 1386.574332] Lustre: server umount lustre-OST0000 complete [ 1389.030125] Lustre: 3666:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780335159/real 1780335159] req@ffff98dc415f1c00 x1866815409435392/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1780335175 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1389.050685] Lustre: 3666:0:(client.c:2480:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1391.151355] Lustre: server umount lustre-OST0001 complete [ 1400.998669] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 13:33:05 (1780335185) [ 1422.624272] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1435.899857] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1442.097789] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1452.517691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1458.270045] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1461.935770] Lustre: 47478:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1471.076578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1479.419096] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1488.953176] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1496.827743] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1505.990912] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1512.304733] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1529.828244] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1529.834278] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1529.844941] Lustre: Skipped 16 previous similar messages [ 1533.236392] Lustre: server umount lustre-MDT0000 complete [ 1536.683588] LustreError: 46339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780335323 with bad export cookie 17518319614054199389 [ 1536.690648] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1536.698261] LustreError: 46339:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1536.989336] Lustre: server umount lustre-MDT0001 complete [ 1550.615903] Lustre: server umount lustre-OST0000 complete [ 1566.505772] Lustre: server umount lustre-OST0001 complete [ 1578.996653] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 13:36:03 (1780335363) [ 1582.627528] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1584.707181] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 13:36:09 (1780335369) [ 1595.757628] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 1607.060281] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1634.798282] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1635.299788] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1635.506848] Lustre: lustre-MDT0000: new disk, initializing [ 1635.756575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1640.317226] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1647.944156] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1655.449545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1655.691467] Lustre: lustre-OST0000: new disk, initializing [ 1655.694812] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1655.698210] Lustre: Skipped 1 previous similar message [ 1655.703962] Lustre: 53373:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1657.737858] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1657.750278] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1657.785445] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1662.038278] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1670.036642] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1672.722763] Lustre: DEBUG MARKER: create test files [ 1709.029327] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1709.043894] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1709.056940] Lustre: Skipped 11 previous similar messages [ 1712.762248] Lustre: server umount lustre-OST0000 complete [ 1716.915616] Lustre: server umount lustre-MDT0000 complete [ 1731.274294] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 1735.342807] Key type lgssc unregistered [ 1735.845170] LNet: 54875:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1735.856769] LNetError: 54875:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1735.871864] LNet: Removed LNI 192.168.202.131@tcp [ 1737.149094] Key type .llcrypt unregistered [ 1737.152443] Key type ._llcrypt unregistered [ 1767.152615] Key type ._llcrypt registered [ 1767.158090] Key type .llcrypt registered [ 1767.255315] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1768.433609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1768.467308] alg: No test for adler32 (adler32-zlib) [ 1769.536514] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 1769.756842] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 1771.431212] Key type lgssc registered [ 1772.627910] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1780.070774] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1780.092862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1781.830388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1786.071571] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1791.075993] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1798.662922] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1798.995804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1805.503969] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1806.188119] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1806.254511] Lustre: 57464:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1806.271971] Lustre: 57464:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1806.277149] Lustre: 57464:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1806.284806] Lustre: 57464:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1806.293830] Lustre: 57464:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1806.297316] Lustre: 57464:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1811.964091] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1818.016648] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1824.632696] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1831.910083] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1831.917517] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1831.928424] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1832.994996] Lustre: server umount lustre-OST0000 complete [ 1836.620238] Lustre: server umount lustre-MDT0000 complete [ 1846.115490] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 1848.471945] Key type lgssc unregistered [ 1848.746771] LNet: 58940:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1848.767259] LNetError: 58940:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1848.792910] LNet: Removed LNI 192.168.202.131@tcp [ 1849.513608] Key type .llcrypt unregistered [ 1849.515663] Key type ._llcrypt unregistered [ 1881.248962] Key type ._llcrypt registered [ 1881.252375] Key type .llcrypt registered [ 1881.342542] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 1882.822680] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1882.883121] alg: No test for adler32 (adler32-zlib) [ 1884.043308] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 1884.253444] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 1885.975191] Key type lgssc registered [ 1887.179588] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1895.754453] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1895.783082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1897.589528] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1902.789154] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1910.289712] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1918.681384] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1919.069090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1922.162401] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1922.304265] Lustre: 61642:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1922.328974] Lustre: 61642:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1922.342078] Lustre: 61642:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1922.362384] Lustre: 61642:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1922.381396] Lustre: 61642:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1922.390478] Lustre: 61642:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1926.373795] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 1933.618816] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1939.272572] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1953.251648] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1953.261073] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1953.272919] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1954.339104] Lustre: server umount lustre-OST0000 complete [ 1959.010965] Lustre: server umount lustre-MDT0000 complete [ 1970.781287] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 1973.102342] Key type lgssc unregistered [ 1973.336392] LNet: 63070:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1973.344396] LNetError: 63070:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1973.367782] LNet: Removed LNI 192.168.202.131@tcp [ 1974.247969] Key type .llcrypt unregistered [ 1974.249825] Key type ._llcrypt unregistered [ 1997.962338] Key type ._llcrypt registered [ 1997.966691] Key type .llcrypt registered [ 1998.060587] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 2015.077807] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 2015.918110] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2015.965620] alg: No test for adler32 (adler32-zlib) [ 2017.057938] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 2017.242799] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 2018.903503] Key type lgssc registered [ 2019.874222] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2058.619701] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2058.650442] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2059.986528] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2060.032519] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2060.171421] Lustre: lustre-MDT0000: new disk, initializing [ 2060.268538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2060.289337] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2064.746122] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2076.532259] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2076.681745] Lustre: 66196:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2076.731725] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2076.735872] Lustre: Skipped 1 previous similar message [ 2076.801711] Lustre: lustre-MDT0001: new disk, initializing [ 2076.889449] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2076.944592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2076.955511] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2080.894953] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2089.277223] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2094.355861] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2102.470757] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2102.831971] Lustre: lustre-OST0000: new disk, initializing [ 2102.835390] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2102.840404] Lustre: 67941:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2102.942133] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2110.001275] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2111.084217] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2111.095801] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2111.240126] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2118.864719] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2123.980651] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2124.263972] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2130.462203] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2130.912408] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2137.060053] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2137.065919] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2137.081873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2137.089078] Lustre: Skipped 1 previous similar message [ 2139.838385] Lustre: server umount lustre-OST0000 complete [ 2147.299131] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.307231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2147.321249] Lustre: Skipped 2 previous similar messages [ 2151.277433] Lustre: server umount lustre-MDT0000 complete [ 2154.896314] LustreError: 66187:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780335941 with bad export cookie 1017072280339191537 [ 2154.899108] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2154.903747] LustreError: 66187:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2155.308150] Lustre: server umount lustre-MDT0001 complete [ 2164.006697] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 13:45:48 (1780335948) [ 2174.544290] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 2222.108409] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 2236.335303] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 2278.289330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2278.579512] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2278.605600] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2278.699955] Lustre: lustre-MDT0000: new disk, initializing [ 2278.811816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2278.828179] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2283.347349] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2295.675978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2295.847808] Lustre: 73765:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2295.976376] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2295.982785] Lustre: Skipped 1 previous similar message [ 2296.134672] Lustre: lustre-MDT0001: new disk, initializing [ 2296.247278] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2296.320900] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2296.343325] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2301.770869] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2310.916153] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2315.536153] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2324.979740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2325.272721] Lustre: lustre-OST0000: new disk, initializing [ 2325.277783] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2325.284200] Lustre: 75509:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2325.404156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2331.251799] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2331.270762] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2331.441606] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2331.813737] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2341.928546] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2348.472271] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2348.814552] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2356.452500] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2356.811182] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2362.349728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2362.366572] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2362.373267] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2362.378043] Lustre: Skipped 1 previous similar message [ 2365.474677] Lustre: server umount lustre-OST0000 complete [ 2372.578404] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2372.588404] Lustre: Skipped 1 previous similar message [ 2372.601067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2372.603582] Lustre: Skipped 1 previous similar message [ 2376.101381] Lustre: server umount lustre-MDT0000 complete [ 2377.702039] LustreError: 73771:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2377.719381] LustreError: 73771:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2380.624224] LustreError: 73756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780336167 with bad export cookie 1017072280339192461 [ 2380.627475] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2380.662892] LustreError: 73756:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2381.156429] Lustre: server umount lustre-MDT0001 complete [ 2390.535288] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 13:49:35 (1780336175) [ 2397.245257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2397.615401] LustreError: 77775:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2397.690843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2402.006136] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2402.787637] LustreError: 77776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.906509] LustreError: 77775:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.096470] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2416.346090] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2422.727231] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2427.005924] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2434.757725] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2435.054602] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2435.063186] Lustre: Skipped 1 previous similar message [ 2441.346419] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2448.571797] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2453.483496] Lustre: 80110:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 12f386ac-f036-4f37-931e-29c7f6be1515 at adminstrative request [ 2457.593204] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2457.618694] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2467.820871] Lustre: MGS: Client 12f386ac-f036-4f37-931e-29c7f6be1515 (at 0@lo) reconnecting [ 2467.846752] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xe1d5dd9ac839e29 to 0xe1d5dd9ac83a0c9 [ 2467.890482] Lustre: MGC192.168.202.131@tcp: Connection restored to 0@lo (at 0@lo) [ 2493.942234] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2494.146137] Lustre: lustre-OST0001: new disk, initializing [ 2494.150398] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2494.156435] Lustre: 80502:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2500.397564] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2501.708722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2501.724610] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2501.869568] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2508.154416] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2512.357281] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2512.359865] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2512.363273] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2512.374157] Lustre: Skipped 2 previous similar messages [ 2512.397158] Lustre: Skipped 1 previous similar message [ 2517.799906] Lustre: server umount lustre-OST0001 complete [ 2522.609209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2522.612080] LustreError: 79375:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2522.634900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2522.634910] Lustre: Skipped 2 previous similar messages [ 2522.648308] Lustre: Skipped 1 previous similar message [ 2522.703268] LustreError: 79375:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2528.056274] Lustre: server umount lustre-OST0000 complete [ 2538.809591] Lustre: server umount lustre-MDT0000 complete [ 2543.073512] LustreError: 77776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2543.091518] LustreError: 77776:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2543.490878] LustreError: 77757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780336330 with bad export cookie 1017072280339194057 [ 2543.491958] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2543.499462] LustreError: 77757:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2543.959380] Lustre: server umount lustre-MDT0001 complete [ 2553.597211] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 13:52:18 (1780336338) [ 2555.566841] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2557.556785] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 13:52:22 (1780336342) [ 2559.011446] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2561.101213] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 13:52:25 (1780336345) [ 2562.793924] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2565.493576] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 13:52:29 (1780336349) [ 2568.213139] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2570.559372] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 13:52:35 (1780336355) [ 2572.828759] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2579.130246] loop: module loaded [ 2579.167578] loop0: detected capacity change from 0 to 4294967296 [ 2579.254343] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2581.536522] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2585.247655] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2589.548918] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2619.665215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2619.721556] LustreError: 84496:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2619.731213] LustreError: 84496:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2619.746449] LustreError: 84496:0:(tgt_mount.c:2537:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2619.763339] LustreError: 84496:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2642.944573] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 2657.019955] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 2702.294647] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2702.867781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2702.934624] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2703.060080] Lustre: lustre-MDT0000: new disk, initializing [ 2703.201226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2703.204147] Lustre: Skipped 1 previous similar message [ 2703.231670] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2709.007459] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2720.698340] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2720.812600] Lustre: 87328:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2721.028921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2725.304045] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2734.005256] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2738.517510] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2746.198207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2746.398076] Lustre: 89075:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2751.638596] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2751.848966] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2754.989351] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2763.881359] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2769.716657] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2769.965053] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2775.264821] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2775.535905] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2782.691057] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2782.700962] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2782.702466] Lustre: Skipped 3 previous similar messages [ 2782.710123] Lustre: Skipped 8 previous similar messages [ 2784.542441] Lustre: server umount lustre-OST0000 complete [ 2792.928416] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2794.998254] Lustre: server umount lustre-MDT0000 complete [ 2798.050674] LustreError: 87339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2798.076717] LustreError: 87339:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2799.186273] LustreError: 87321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780336586 with bad export cookie 1017072280339194673 [ 2799.188908] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2799.194942] LustreError: 87321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2799.723486] Lustre: server umount lustre-MDT0001 complete [ 2809.790491] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 13:56:34 (1780336594) [ 2817.233053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2822.942963] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2832.363777] LustreError: 91340:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2832.408983] LustreError: 91340:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2833.632521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2834.044930] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2834.050334] Lustre: Skipped 3 previous similar messages [ 2839.723980] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2846.417610] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2851.398916] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2859.088878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2865.983873] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2872.730530] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2878.612942] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2878.875660] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2884.195485] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2884.449849] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2892.500745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2892.664747] Lustre: lustre-OST0001: new disk, initializing [ 2892.669383] Lustre: Skipped 2 previous similar messages [ 2892.672800] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2892.676338] Lustre: Skipped 3 previous similar messages [ 2892.681371] Lustre: 94342:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2898.568038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2898.578532] Lustre: Skipped 2 previous similar messages [ 2898.589695] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2898.693717] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2899.343394] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 2907.474197] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2912.131587] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2912.374164] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2916.797451] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2916.955815] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2938.186283] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2965.037718] Lustre: Failing over lustre-MDT0000 [ 2965.455625] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.31@tcp (stopping) [ 2965.459215] Lustre: Skipped 2 previous similar messages [ 2965.495018] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2965.514553] Lustre: Skipped 4 previous similar messages [ 2967.669854] Lustre: server umount lustre-MDT0000 complete [ 2970.521489] LustreError: 95447:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2970.562341] LustreError: 95447:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2986.979661] Lustre: 64418:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1780336757/real 1780336757] req@00000000c14bc565 x1866817396307456/t0(0) o400->MGC192.168.202.131@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1780336773 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2987.054615] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2999.948639] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2999.951849] LDISKFS-fs (dm-0): recovery complete [ 2999.961816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3013.110333] LustreError: 64416:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000729b811e x1866817396319872/t0(0) o250->MGC192.168.202.131@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3013.812293] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3017.218144] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3017.220774] Lustre: 92058:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3019.242833] Lustre: 91341:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3019.243962] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3019.251514] Lustre: 91341:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 3019.429102] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3019.765634] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3020.143306] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3020.170934] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 3020.275433] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3020.519086] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3020.529546] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 3020.550930] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3020.570793] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 3021.218265] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3021.238248] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 3021.591113] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3021.597792] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 3022.618049] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3022.639150] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 5 previous similar messages [ 3023.322502] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3023.443159] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 3023.658042] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3023.674493] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 3026.923497] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3026.974067] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 3027.635724] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3027.655277] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 3027.991662] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3028.017170] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 3035.064103] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3035.079663] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 24 previous similar messages [ 3035.728383] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3035.741102] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 3036.072619] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3036.087102] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 3051.363509] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3051.379169] Lustre: 97045:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 50 previous similar messages [ 3052.030902] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3052.049527] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 3052.384189] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3052.399755] LustreError: 97045:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 3077.153912] Lustre: 97045:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3077.166182] Lustre: 97045:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3077.176048] Lustre: 97045:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 13c97261-d19c-4f76-9b4d-1e923a4f731b@192.168.202.31@tcp [ 3077.183548] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3077.191358] Lustre: 97045:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@0000000017465cde x1866817378260480/t0(8589934780) o35->13c97261-d19c-4f76-9b4d-1e923a4f731b@192.168.202.31@tcp:64/0 lens 392/0 e 0 to 0 dl 1780336874 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3077.216260] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3077.230732] Lustre: lustre-MDT0000: Denying connection for new client 13c97261-d19c-4f76-9b4d-1e923a4f731b (at 192.168.202.31@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3077.258758] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3077.327254] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3077.330896] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3077.339958] Lustre: Skipped 2 previous similar messages [ 3077.370966] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3077.382940] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3093.479727] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3093.488381] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3093.512043] Lustre: Skipped 1 previous similar message [ 3093.519577] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3093.526145] Lustre: Skipped 5 previous similar messages [ 3099.268348] Lustre: server umount lustre-OST0000 complete [ 3101.160363] LustreError: 92941:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3101.189758] LustreError: 92941:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 63 previous similar messages [ 3103.713261] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3103.726673] LustreError: Skipped 2 previous similar messages [ 3109.570750] Lustre: server umount lustre-OST0001 complete [ 3119.772189] Lustre: server umount lustre-MDT0000 complete [ 3124.140865] LustreError: 91321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780336911 with bad export cookie 1017072280339245934 [ 3124.141673] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3124.166159] LustreError: 91321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3125.107409] Lustre: server umount lustre-MDT0001 complete [ 3137.973616] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 14:02:02 (1780336922) [ 3147.570513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3147.577528] Lustre: *** cfs_fail_loc=197, val=0*** [ 3163.169821] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3168.351473] LustreError: 99018:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.131@tcp: failed processing log, type 4: rc = -110 [ 3173.490420] Lustre: *** cfs_fail_loc=197, val=0*** [ 3173.493803] Lustre: Skipped 65 previous similar messages [ 3194.079339] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3194.087134] Lustre: Skipped 3 previous similar messages [ 3200.967827] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3207.717464] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3210.278914] Lustre: server umount lustre-OST0000 complete [ 3219.915539] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 14:03:24 (1780337004) [ 3243.075348] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3248.653711] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3257.792764] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3262.886157] LustreError: 100564:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3262.904432] LustreError: 100564:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3262.917978] LustreError: 100564:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3263.629556] Lustre: server umount lustre-MDT0000 complete [ 3304.978156] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 3315.416976] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 3356.884782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3357.119650] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3357.137478] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3357.222388] Lustre: lustre-MDT0000: new disk, initializing [ 3357.436484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3361.747042] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3374.380345] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3374.530770] Lustre: 104593:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3374.891251] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3380.009729] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3387.876444] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3392.585584] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3400.170853] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3400.361645] Lustre: 106338:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3407.467569] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3409.125392] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3415.591663] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3420.230822] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3420.491873] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3424.646534] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3424.881996] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3429.869122] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3429.887831] Lustre: Skipped 5 previous similar messages [ 3429.910995] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3429.915502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3429.925348] LustreError: Skipped 1 previous similar message [ 3429.940313] Lustre: Skipped 11 previous similar messages [ 3441.119131] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3441.210624] Lustre: server umount lustre-OST0000 complete [ 3451.295628] Lustre: server umount lustre-MDT0000 complete [ 3455.466189] LustreError: 104603:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3455.508176] LustreError: 104603:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 9 previous similar messages [ 3455.579127] LustreError: 104585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780337242 with bad export cookie 1017072280339256588 [ 3455.584674] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3455.589351] LustreError: 104585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3456.222305] Lustre: server umount lustre-MDT0001 complete [ 3466.612759] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 14:07:31 (1780337251) [ 3476.395901] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 3479.393495] Key type lgssc unregistered [ 3479.689630] LNet: 108661:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3479.708068] LNetError: 108661:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3479.750803] LNet: Removed LNI 192.168.202.131@tcp [ 3480.684344] Key type .llcrypt unregistered [ 3480.687123] Key type ._llcrypt unregistered [ 3500.123540] Key type ._llcrypt registered [ 3500.127290] Key type .llcrypt registered [ 3500.234335] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 3515.744319] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 3517.064960] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3517.106910] alg: No test for adler32 (adler32-zlib) [ 3518.138845] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 3518.401955] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 3520.143192] Key type lgssc registered [ 3521.300261] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3564.243304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3564.269755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3565.554535] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3565.589687] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3565.642581] Lustre: lustre-MDT0000: new disk, initializing [ 3565.703930] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3565.731944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3570.360025] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3581.228487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3581.445598] Lustre: lustre-OST0000: new disk, initializing [ 3581.449532] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3581.453693] Lustre: Skipped 1 previous similar message [ 3581.457976] Lustre: 112784:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3581.520740] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3587.711793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3587.722710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3587.760524] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3588.069230] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3598.852117] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3598.980688] Lustre: lustre-OST0001: new disk, initializing [ 3598.984655] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3598.993382] Lustre: 113629:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3599.052318] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3605.076282] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3606.151450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3606.160153] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3606.188800] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3616.224703] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3616.263715] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3616.275404] Lustre: Skipped 1 previous similar message [ 3620.379497] Lustre: server umount lustre-MDT0000 complete [ 3627.280491] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3631.570662] LustreError: 113628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780337418 with bad export cookie 15531189232735041996 [ 3631.580329] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3631.592814] LustreError: 113628:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3631.688044] Lustre: server umount lustre-OST0000 complete [ 3635.572417] Lustre: server umount lustre-OST0001 complete [ 3644.676938] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 14:10:29 (1780337429) [ 3654.654752] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 3682.555700] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 14:11:07 (1780337467) [ 3727.753713] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 3738.402921] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 3776.872505] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3777.210961] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3777.276419] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3777.358652] Lustre: lustre-MDT0000: new disk, initializing [ 3777.429528] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3777.450214] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3781.797628] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3792.440317] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3792.578730] Lustre: 120314:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3792.635596] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3792.641898] Lustre: Skipped 1 previous similar message [ 3792.720304] Lustre: lustre-MDT0001: new disk, initializing [ 3792.858846] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3792.926993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3792.944902] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3797.261898] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3804.805987] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3808.477383] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3815.182393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3815.437673] Lustre: lustre-OST0000: new disk, initializing [ 3815.440986] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3815.447228] Lustre: 122058:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3815.531236] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3821.192685] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3822.180769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3822.189575] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3822.333785] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3828.558084] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3832.793105] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3832.999558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3837.001688] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3837.216140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3843.042885] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3843.047680] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3843.050609] Lustre: Skipped 1 previous similar message [ 3843.052241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3845.638207] Lustre: server umount lustre-OST0000 complete [ 3853.294280] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3853.299588] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3853.307702] Lustre: Skipped 2 previous similar messages [ 3853.314152] Lustre: Skipped 1 previous similar message [ 3856.198662] Lustre: server umount lustre-MDT0000 complete [ 3861.614193] LustreError: 121100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780337648 with bad export cookie 15531189232735043235 [ 3861.639753] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3862.369942] Lustre: server umount lustre-MDT0001 complete [ 3873.166170] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 14:14:17 (1780337657) [ 3880.433787] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3880.990755] LustreError: 124326:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3881.014462] LustreError: 124326:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3881.059206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3886.058045] LustreError: 124326:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3886.079126] LustreError: 124326:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3886.302332] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3890.147404] LustreError: 124327:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3895.269749] LustreError: 124326:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3896.231729] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3900.803624] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3906.964304] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3911.610474] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3918.591364] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3918.955144] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3918.966850] Lustre: Skipped 1 previous similar message [ 3926.354803] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 3932.296901] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3936.722960] Lustre: *** cfs_fail_loc=159, val=0*** [ 3953.146123] Lustre: lustre-MDT0000: Client 48ad5592-658a-4975-bcf6-45e787900154 (at 192.168.202.31@tcp) reconnecting [ 3954.796532] Lustre: *** cfs_fail_loc=159, val=0*** [ 3954.803654] Lustre: Skipped 5 previous similar messages [ 3971.040955] Lustre: lustre-MDT0000: Client 48ad5592-658a-4975-bcf6-45e787900154 (at 192.168.202.31@tcp) reconnecting [ 3977.697794] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3977.700861] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3977.712712] LustreError: Skipped 1 previous similar message [ 3977.748795] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3977.760690] Lustre: Skipped 2 previous similar messages [ 3978.439495] Lustre: server umount lustre-OST0000 complete [ 3983.841796] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3983.849474] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3983.862431] Lustre: Skipped 1 previous similar message [ 3983.874224] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3989.353156] Lustre: server umount lustre-MDT0000 complete [ 3992.843066] LustreError: 126592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780337779 with bad export cookie 15531189232735044159 [ 3992.849499] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3992.857954] LustreError: 126592:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 3993.156989] Lustre: server umount lustre-MDT0001 complete [ 4004.802279] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4007.468384] Key type lgssc unregistered [ 4007.733907] LNet: 127746:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4007.749307] LNetError: 127746:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4007.764789] LNet: Removed LNI 192.168.202.131@tcp [ 4008.594872] Key type .llcrypt unregistered [ 4008.596962] Key type ._llcrypt unregistered [ 4036.057281] Key type ._llcrypt registered [ 4036.063565] Key type .llcrypt registered [ 4036.236107] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 4037.430457] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4037.601351] alg: No test for adler32 (adler32-zlib) [ 4038.696734] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 4038.946352] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 4040.631208] Key type lgssc registered [ 4042.027931] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4049.812674] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4049.836452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4051.411497] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4056.028507] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4061.666422] LustreError: 129143:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4061.686329] LustreError: 129143:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4065.399559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4065.685528] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4070.929726] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4077.741755] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4082.743667] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4090.773711] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4091.073764] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4096.248052] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4098.674445] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4107.162467] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4121.256250] Lustre: *** cfs_fail_loc=159, val=0*** [ 4136.903227] Lustre: lustre-MDT0000: Client a404c64f-ad2b-44f3-8906-172bb6d49524 (at 192.168.202.31@tcp) reconnecting [ 4139.841325] Lustre: *** cfs_fail_loc=159, val=0*** [ 4156.376844] Lustre: lustre-MDT0001: Client a404c64f-ad2b-44f3-8906-172bb6d49524 (at 192.168.202.31@tcp) reconnecting [ 4157.887149] Lustre: *** cfs_fail_loc=159, val=0*** [ 4157.888567] Lustre: Skipped 3 previous similar messages [ 4174.293737] Lustre: lustre-MDT0001: Client a404c64f-ad2b-44f3-8906-172bb6d49524 (at 192.168.202.31@tcp) reconnecting [ 4179.695226] Lustre: 131844:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4183.012582] Lustre: *** cfs_fail_loc=159, val=0*** [ 4183.019692] Lustre: Skipped 3 previous similar messages [ 4198.839911] Lustre: lustre-MDT0000: Client 32d9d652-970f-4bdb-9d6c-e812f9ac13d7 (at 192.168.202.31@tcp) reconnecting [ 4201.253993] Lustre: *** cfs_fail_loc=159, val=0*** [ 4201.255744] Lustre: Skipped 14 previous similar messages [ 4216.794655] Lustre: lustre-MDT0000: Client 32d9d652-970f-4bdb-9d6c-e812f9ac13d7 (at 192.168.202.31@tcp) reconnecting [ 4218.657502] Lustre: 132146:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4224.484625] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4224.495783] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4224.504458] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4225.002655] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4225.014165] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4225.018967] LustreError: Skipped 1 previous similar message [ 4225.047849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4228.578601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4228.587832] Lustre: Skipped 1 previous similar message [ 4230.706285] Lustre: server umount lustre-OST0000 complete [ 4234.723523] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4234.732898] LustreError: Skipped 3 previous similar messages [ 4234.738309] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4234.754548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4238.816159] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4238.816501] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4241.061658] Lustre: server umount lustre-MDT0000 complete [ 4243.939240] LustreError: 129855:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4243.956285] LustreError: 129855:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4245.710601] LustreError: 131966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338032 with bad export cookie 2348475464756246703 [ 4245.716113] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4245.726153] LustreError: 131966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4246.187952] Lustre: server umount lustre-MDT0001 complete [ 4258.805549] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4261.369719] Key type lgssc unregistered [ 4261.739245] LNet: 133183:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4261.750866] LNetError: 133183:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4262.821957] LNet: Removed LNI 192.168.202.131@tcp [ 4263.777710] Key type .llcrypt unregistered [ 4263.789427] Key type ._llcrypt unregistered [ 4292.470299] Key type ._llcrypt registered [ 4292.475036] Key type .llcrypt registered [ 4292.586395] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 4294.801075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4294.878276] alg: No test for adler32 (adler32-zlib) [ 4296.197162] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 4296.564912] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 4298.455374] Key type lgssc registered [ 4300.024569] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4309.843669] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4309.867181] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4311.516646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4317.048352] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4321.762532] LustreError: 134581:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4321.787937] LustreError: 134581:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4326.881319] LustreError: 134582:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4328.089174] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4328.708481] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4334.088869] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4340.359962] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4346.066857] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4353.874753] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4354.252652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4356.346183] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4362.282759] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4369.367853] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4373.750058] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4379.540033] Lustre: 136830:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4383.487723] Lustre: 136879:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4386.137196] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4387.445382] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4393.413878] Lustre: 137027:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4394.755366] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4396.032383] Lustre: 137135:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4402.658145] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4402.683573] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4402.714754] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4404.706067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4404.708528] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4404.720789] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4407.156484] Lustre: server umount lustre-OST0000 complete [ 4414.952792] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4414.960686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4414.986855] Lustre: Skipped 1 previous similar message [ 4415.007204] Lustre: Skipped 2 previous similar messages [ 4418.383322] Lustre: server umount lustre-MDT0000 complete [ 4420.068281] LustreError: 134582:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4420.100083] LustreError: 134582:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4423.553472] LustreError: 137441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338210 with bad export cookie 11121729242993004626 [ 4423.557542] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4423.570908] LustreError: 137441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4424.217580] Lustre: server umount lustre-MDT0001 complete [ 4436.192000] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4438.896246] Key type lgssc unregistered [ 4439.242748] LNet: 138176:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4439.251150] LNetError: 138176:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4439.272806] LNet: Removed LNI 192.168.202.131@tcp [ 4440.401123] Key type .llcrypt unregistered [ 4440.404731] Key type ._llcrypt unregistered [ 4465.819311] Key type ._llcrypt registered [ 4465.821074] Key type .llcrypt registered [ 4465.949030] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 4467.805085] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4467.851144] alg: No test for adler32 (adler32-zlib) [ 4469.073349] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 4469.580817] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 4471.383176] Key type lgssc registered [ 4473.214846] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4482.990167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4483.015866] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4484.564572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4489.030666] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4494.819513] LustreError: 139577:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4494.831405] LustreError: 139577:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4497.649596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4498.139370] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4502.767485] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4510.504737] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4515.959898] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4525.682156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4526.322407] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4529.408146] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4533.919343] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4541.319237] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4542.734329] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4547.478722] Lustre: *** cfs_fail_loc=159, val=0*** [ 4562.908861] Lustre: lustre-MDT0000: Client d209518b-476e-4265-9d55-96b9a67075dc (at 192.168.202.31@tcp) reconnecting [ 4568.550150] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4568.551597] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4568.573155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4568.581894] Lustre: Skipped 1 previous similar message [ 4568.593927] Lustre: Skipped 1 previous similar message [ 4569.749829] Lustre: server umount lustre-OST0000 complete [ 4574.689890] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4574.715795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4574.767148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4578.785404] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4578.788810] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4578.807722] Lustre: Skipped 1 previous similar message [ 4580.340855] Lustre: server umount lustre-MDT0000 complete [ 4583.906677] LustreError: 139571:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4583.921899] LustreError: 139571:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4585.373260] LustreError: 139558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338372 with bad export cookie 18084803638061676922 [ 4585.378773] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4585.393419] LustreError: 139558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4585.805195] Lustre: server umount lustre-MDT0001 complete [ 4598.608755] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4601.696327] Key type lgssc unregistered [ 4601.929580] LNet: 142890:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4601.937768] LNetError: 142890:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4601.976082] LNet: Removed LNI 192.168.202.131@tcp [ 4602.709903] Key type .llcrypt unregistered [ 4602.713661] Key type ._llcrypt unregistered [ 4629.528673] Key type ._llcrypt registered [ 4629.531031] Key type .llcrypt registered [ 4629.630547] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 4631.297682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4631.611289] alg: No test for adler32 (adler32-zlib) [ 4632.731263] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 4632.992803] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 4634.720480] Key type lgssc registered [ 4635.990452] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4644.021635] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4644.044774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4645.593754] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4650.695159] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4656.107869] LustreError: 144306:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4656.124933] LustreError: 144306:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4658.780419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4659.122191] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4663.652402] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4669.448043] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4673.602912] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4680.439942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4680.884523] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4683.961782] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4686.243941] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4691.190550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4692.371505] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4714.366684] Lustre: lustre-MDT0001: evicting b3e6b113-d06e-4bb8-b787-49a4569881ea (at 192.168.202.31@tcp) by administrative request [ 4714.375798] Lustre: Skipped 1 previous similar message [ 4745.698751] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4745.714390] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4745.725166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4747.747718] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4747.759962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4747.779738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4749.529491] Lustre: server umount lustre-OST0000 complete [ 4756.451342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4756.458142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4756.473731] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4756.490014] Lustre: Skipped 1 previous similar message [ 4759.917430] Lustre: server umount lustre-MDT0000 complete [ 4763.114509] LustreError: 144305:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4763.126768] LustreError: 144305:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4763.659357] LustreError: 144286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338550 with bad export cookie 14846606683321174219 [ 4763.672805] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4763.677225] LustreError: 144286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4764.029214] Lustre: server umount lustre-MDT0001 complete [ 4780.379332] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4783.830071] Key type lgssc unregistered [ 4784.209984] LNet: 148019:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4784.223138] LNetError: 148019:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4784.241511] LNet: Removed LNI 192.168.202.131@tcp [ 4785.364908] Key type .llcrypt unregistered [ 4785.367375] Key type ._llcrypt unregistered [ 4861.640610] Key type ._llcrypt registered [ 4861.648337] Key type .llcrypt registered [ 4861.809577] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 4863.689651] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4863.770404] alg: No test for adler32 (adler32-zlib) [ 4864.873702] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 4865.193752] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 4866.943370] Key type lgssc registered [ 4868.758048] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4878.780132] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4878.811851] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4880.209064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4884.420453] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4890.596307] LustreError: 150500:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4890.616911] LustreError: 150500:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4893.261569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4893.655562] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4898.498902] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4904.714302] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4909.337348] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4917.750279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4918.136578] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4919.186934] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4924.845881] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 4925.291633] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4932.052720] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4940.768940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4940.780394] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4940.785253] Lustre: Skipped 1 previous similar message [ 4945.899929] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4945.911654] Lustre: Skipped 2 previous similar messages [ 4946.547504] Lustre: server umount lustre-OST0000 complete [ 4955.104079] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4955.111542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4955.137150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4956.130265] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4957.876987] Lustre: server umount lustre-MDT0000 complete [ 4961.252147] LustreError: 151216:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4961.277845] LustreError: 151216:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4961.949447] LustreError: 150482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338748 with bad export cookie 18283057561469678130 [ 4961.965278] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4961.966898] LustreError: 150482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4962.339757] Lustre: server umount lustre-MDT0001 complete [ 4974.094655] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 4976.568311] Key type lgssc unregistered [ 4976.872723] LNet: 153687:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4976.877883] LNetError: 153687:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4976.895471] LNet: Removed LNI 192.168.202.131@tcp [ 4977.593689] Key type .llcrypt unregistered [ 4977.598202] Key type ._llcrypt unregistered [ 5000.212502] Key type ._llcrypt registered [ 5000.215711] Key type .llcrypt registered [ 5000.314958] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 5001.078392] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5001.096292] alg: No test for adler32 (adler32-zlib) [ 5002.218788] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 5002.398430] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 5004.087891] Key type lgssc registered [ 5005.293795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5021.385887] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 14:33:26 (1780338806) [ 5037.446162] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 5047.317404] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 5087.331105] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5087.356880] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5088.738964] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5088.793616] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5088.905243] Lustre: lustre-MDT0000: new disk, initializing [ 5089.030868] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5089.067960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5093.904805] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5104.565249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5104.762925] Lustre: 157902:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5104.892387] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5104.902855] Lustre: Skipped 1 previous similar message [ 5105.039757] Lustre: lustre-MDT0001: new disk, initializing [ 5105.168591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5105.210643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5105.227789] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5110.624642] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5119.294529] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5123.701239] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5130.943623] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5131.263600] Lustre: lustre-OST0000: new disk, initializing [ 5131.271663] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5131.284949] Lustre: 159644:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5131.398104] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5137.790180] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5140.602181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5140.625583] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5140.766838] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5145.978258] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5161.470801] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5161.506213] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5161.512116] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5161.529851] Lustre: Skipped 1 previous similar message [ 5163.184691] Lustre: server umount lustre-OST0000 complete [ 5171.690154] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5171.698059] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5171.708047] Lustre: Skipped 1 previous similar message [ 5171.716290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5171.724455] Lustre: Skipped 1 previous similar message [ 5174.532759] Lustre: server umount lustre-MDT0000 complete [ 5179.512140] LustreError: 157894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780338966 with bad export cookie 9654231932806653379 [ 5179.520682] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5179.523597] LustreError: 157894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5180.100157] Lustre: server umount lustre-MDT0001 complete [ 5190.959789] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 5194.116433] Key type lgssc unregistered [ 5194.524311] LNet: 161420:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5194.529790] LNetError: 161420:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5194.554037] LNet: Removed LNI 192.168.202.131@tcp [ 5195.395825] Key type .llcrypt unregistered [ 5195.398068] Key type ._llcrypt unregistered [ 5222.314566] Key type ._llcrypt registered [ 5222.315943] Key type .llcrypt registered [ 5222.403202] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 5223.213555] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5223.251359] alg: No test for adler32 (adler32-zlib) [ 5224.294714] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 5224.437732] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 5226.112184] Key type lgssc registered [ 5227.757568] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5240.446400] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5240.471742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5242.462824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5248.387791] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5252.598868] LustreError: 162820:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5252.626940] LustreError: 162820:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5257.695726] LustreError: 162819:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5258.488499] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5258.742334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5263.736777] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5270.713366] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5275.197584] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5282.105552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5282.537311] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5287.665402] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5288.662885] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 5295.390604] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5302.101305] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5302.119258] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5302.128179] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5302.133337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.148403] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5303.170883] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5303.200313] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5303.219622] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5303.233446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.257793] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5304.400287] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5304.412426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5304.426921] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5304.435474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.449531] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5305.225316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.459120] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5306.472539] Lustre: Skipped 1 previous similar message [ 5306.499365] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5306.510381] Lustre: Skipped 1 previous similar message [ 5306.514094] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5306.517156] Lustre: Skipped 1 previous similar message [ 5306.521863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.531045] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5306.541732] Lustre: Skipped 1 previous similar message [ 5307.362216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.303666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.228721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.579474] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5310.582284] Lustre: Skipped 3 previous similar messages [ 5310.615918] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5310.631792] Lustre: Skipped 3 previous similar messages [ 5310.635095] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5310.639453] Lustre: Skipped 3 previous similar messages [ 5310.643037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.667890] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5310.670394] Lustre: Skipped 3 previous similar messages [ 5311.741327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.679738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.863686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.869990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.987625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.089669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.951819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.263873] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5319.266425] Lustre: Skipped 7 previous similar messages [ 5319.275544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5319.291798] Lustre: Skipped 7 previous similar messages [ 5319.300808] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5319.306861] Lustre: Skipped 7 previous similar messages [ 5319.309986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.320751] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5319.329896] Lustre: Skipped 7 previous similar messages [ 5320.291219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.363991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.487341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.638310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.812509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.186559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.423260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.446832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.173826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.290086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.285313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.212207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.118768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.202090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.266346] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5336.271722] Lustre: Skipped 14 previous similar messages [ 5336.290930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5336.298074] Lustre: Skipped 14 previous similar messages [ 5336.300833] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5336.304686] Lustre: Skipped 14 previous similar messages [ 5336.306960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.316916] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5336.326364] Lustre: Skipped 14 previous similar messages [ 5337.121503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.129129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.894695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.884299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.866165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.960828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.686090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.709297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.680715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.727721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.523344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.584981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.437102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.346974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.187481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.389256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.364987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.392402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.305290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.457285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.447897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.571906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.444601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.519048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.498330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.636255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.309700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.343455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.502832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.500875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.812811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.797355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.810621] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5368.820893] Lustre: Skipped 32 previous similar messages [ 5368.837641] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5368.859607] Lustre: Skipped 32 previous similar messages [ 5368.863330] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5368.868799] Lustre: Skipped 32 previous similar messages [ 5368.882777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.889149] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5368.891961] Lustre: Skipped 32 previous similar messages [ 5369.811180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.811857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.720952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.559655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.419495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.478271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.562129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.451709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.428658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.532908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.679513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.712726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.910354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.163262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.250487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.521551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.672755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.675753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.790800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.780152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.985779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.166352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.147388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.391874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.679299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.851860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.115591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.290694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.222730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.389596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.448156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.269815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.476469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.656154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.985440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.192222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.386299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.438121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.492519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.533926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.814531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.893023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.977956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.212641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.368234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.318761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.451893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.596613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.684453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.546325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.579751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.632496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.712514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.594266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.682947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.621797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.497836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.459857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.540290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.443101] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5433.447208] Lustre: Skipped 59 previous similar messages [ 5433.458850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5433.465458] Lustre: Skipped 59 previous similar messages [ 5433.468210] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5433.472607] Lustre: Skipped 59 previous similar messages [ 5433.479742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.499210] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5433.509800] Lustre: Skipped 59 previous similar messages [ 5434.609817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.898464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.921088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.322278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.640546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.824663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.155077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.296487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.907671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.057484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.969684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.866810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.015274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.341750] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5450.356365] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5450.387689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.533960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.812333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.947727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.008581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.213061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.417797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.482627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.669393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.998458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.047864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.254049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.556839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.570133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.646904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.682353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.981903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.126267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.292255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.441079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.682226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.794559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.079579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.100271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.092310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.855935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.908897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.462629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.441203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.302927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.225265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.359777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.507103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.452053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.461351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.709680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.780886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.684525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.489691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.633377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.511620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.781200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.860482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.224362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.465903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.466282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.493954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.575324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.837282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.918539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.322326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.624795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.813879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.850750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.158756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.475424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.588335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.670912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.609303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.907412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.984386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.224929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.576439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.488335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.719596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.860270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.928545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.904863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.854576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.990824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.162441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.302871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.553431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.690887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.824631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.076778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.047624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.479601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.673715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.999862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.198194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.387273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.612834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.984598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.949554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.206777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.024426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.029988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.997093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.976441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.988291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.023988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.131499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.073077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.032033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.954682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.977605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.001027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.768456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.704469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.670077] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5561.673954] Lustre: Skipped 113 previous similar messages [ 5561.686776] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5561.695921] Lustre: Skipped 113 previous similar messages [ 5561.701525] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5561.706491] Lustre: Skipped 113 previous similar messages [ 5561.709977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.717301] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5561.720540] Lustre: Skipped 113 previous similar messages [ 5562.701762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.764601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.577600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.584147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.588732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.625382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.611305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.591558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.542327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.441762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.495347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.521797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.404686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.372804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.411818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.259116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.146903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.131279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.068928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.277904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.116475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.124153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.039394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.167414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.058868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.103406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.090636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.263167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.247930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.367806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.198881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.808795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.845359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.204908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.141694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.424115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.808300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.627697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.641498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.577339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.649277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.718746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.637933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.727855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.929481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.816628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.930331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.049346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.962313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.228512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.143657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.395230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.509927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.731449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.687268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.688613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.751071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.780635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.038557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.472344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.535358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.526583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.580197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.599344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.686712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.830706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.883253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.784973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.787248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.834596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.859683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.962330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.265038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.697769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.052822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.094138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.327621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.483740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.764207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.914971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.924597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.184929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.049582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.264433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.463749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.817644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.939421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.042838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.153971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.001203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.133696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.276790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.438452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.642063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.814530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.025964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.099538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.101442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.591112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.732706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.873655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.920113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.353185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.807830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.005455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.242174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.428414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.708690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.727480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.139195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.081741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.039927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.123943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.035357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.080925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.055515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.272473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.436993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.316351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.346247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.236737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.321458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.339851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.226994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.299760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.472361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.650525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.714427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.999228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.300937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.367031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.554549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.589314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.698700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.983978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.932802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.028681] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5711.038258] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5711.055704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.108221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.030451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.070973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.137678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.214868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.150488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.426316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.480756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.520806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.537185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.415312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.338872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.205516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.479557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.721917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.956253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.971325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.154177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.271513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.325930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.603836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.113641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.506746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.744323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.826216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.475924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.015142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.759402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.941609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.312032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.303217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.508796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.461535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.451936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.083916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.080461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.024265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.062178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.237179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.297992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.452483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.689611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.114143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.507991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.874868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.231954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.181721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.422894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.877367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.279918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.501158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.723773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.150149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.479060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.585206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.424399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.532136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.739804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.033612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.419691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.937950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.434108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.681169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.027039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.627265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.863992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.151195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.798234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.121073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.842799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.216831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.069310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.344552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.277274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.434409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.516206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.511279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.379286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.198845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.162391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.507478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.702260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.735917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.963504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.853564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.783576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.704589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.625312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.561601] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5818.565564] Lustre: Skipped 225 previous similar messages [ 5818.579986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5818.593137] Lustre: Skipped 225 previous similar messages [ 5818.596133] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5818.599416] Lustre: Skipped 225 previous similar messages [ 5818.603465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.612277] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5818.618859] Lustre: Skipped 225 previous similar messages [ 5819.674205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.508280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.516514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.338858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.403375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.406601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.256182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.103241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.132840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.966618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.862621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.744857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.675535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.649531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.617583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.510489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.387387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.391454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.200132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.347938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.150419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.136253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.997176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.136351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.100518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.127906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.075757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.170597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.114663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.159943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.070439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.097659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.923784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.036594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.957900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.999780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.744425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.654091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.626401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.799624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.850913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.790385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.751924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.581733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.636024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.558437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.486133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.430245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.371333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.386077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.352807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.342871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.367528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.258509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.272923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.236415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.373466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.501157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.367164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.441131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.368279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.235106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.214451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.334369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.472887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.583793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.852088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.261300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.482962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.702649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.948857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.055139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.206049] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5891.213724] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5891.226472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.503354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.609606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.751994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.222447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.432065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.385553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.665309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.684762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.819734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.744088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.824987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.081861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.432865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.475620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.444825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.470450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.463605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.553304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.639990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.632340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.727679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.693697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.952941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.887685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.913894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.867399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.056561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.970736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.116761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.057341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.159434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.185703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.157955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.016929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.083487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.896934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.937829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.954761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.123319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.331786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.663157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.851750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.084371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.172439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.127877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.190889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.161771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.012768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.017526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.129235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.114410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.857670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.824431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.758792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.756748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.565537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.555000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.424373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.416127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.239522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.148458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.916538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.846461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.868777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.724367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.652602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.650909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.677191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.785537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.691551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.644329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.622110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.878595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.069723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.135486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.088086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.984490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.954960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.820076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.594898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.507131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.270106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.160685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.932802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.873920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.780292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.656156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.633753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.785061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.829066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.825820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.973543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.177120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.211548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.368293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.336167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.363482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.426919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.582508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.682399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.907234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.189844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.307138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.331130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.556224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.646518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.544761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.499156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.412651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.493986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.376468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.391896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.345775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.331480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.260528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.138791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.070597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.987351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.989803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.871185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.843402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.945664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.032267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.995189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.807732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.820760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.118314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.142805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.442083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.624604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.537728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.637928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.760401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.747879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.730239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.734567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.836672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.801894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.991074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.269758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.389213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.523876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.768443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.823262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.963853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.938283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.909738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.000371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.013480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.972073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.958941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.824276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.890862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.014433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.069545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.038654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.189081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.206278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.282769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.580073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.448100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.570589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.469876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.549451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.551199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.693775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.901862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.135476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.132156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.174168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.248237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.139262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.330780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.340946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.517708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.711154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.976791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.915158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.768283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.701178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.424328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.713990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.692632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.902537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.975607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.172205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.213810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.441829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.512032] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6088.524857] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6088.548088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.686660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.004926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.146764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.158302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.293617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.739226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.348607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.880253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.799774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.898597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.150627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.071684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.034515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.773209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.713870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.665407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.516286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.305603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.394596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.689045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.929785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.999889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.746505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.388940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.920862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.262100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.316882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.599469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.794181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.646870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.597599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.465333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.340043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.185124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.246622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.246827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.442853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.651527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.997359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.206070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.277371] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6133.283929] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6133.295263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.202168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.468355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.678388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.002518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.040881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.271241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.287631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.214538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.984850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.109469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.183485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.364471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.531897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.545992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.999869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.163228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.548584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.762796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.773295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.907257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.629444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.563982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.223904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.145507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.753470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.498655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.701365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.846124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.977665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.033550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.124586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.898893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.996021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.662494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.525585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.335780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.325961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.132416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.071056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.128207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.034055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.895759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.806804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.925872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.878057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.972525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.822112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.695917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.513742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.426963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.319549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.171815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.128073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.147061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.150064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.095364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.915064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.873062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.102175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.398866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.576463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.724316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.548265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.616944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.627951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.547385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.397494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.531055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.566843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.928839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.054770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.486863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.491749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.784950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.152460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.467913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.695505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.491349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.446629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.296570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.356639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.317292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.466500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.501559] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6219.511494] LustreError: 162840:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6219.525579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.557545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.627719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.500631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.483960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.342560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.284977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.281051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.281309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.242111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.346379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.421606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.379721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.423691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.421793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.413701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.453265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.450379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.424438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.452657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.662456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.535699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.611281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.556088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.482060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.691751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.599993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.554279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.359497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.315509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.271232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.302445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.385782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.605803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.534323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.697896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.646612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.921846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.098811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.271515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.270370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.191808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.145528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.124399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.304167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.324469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.353483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.582624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.790976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.890506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.901062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.877344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.991622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.183472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.504657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.812810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.304154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.709976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.965079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.205566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.435954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.489925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.793914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.162989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.488804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.028329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.250153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.504903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.461891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.527132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.855935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.057797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.982670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.491333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.817695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.193693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.213169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.360227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.266386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.449877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.193372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.188820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.219434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.458976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.409108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.314857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.356266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.401268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.277737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.501937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.774618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.001760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.123808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.933694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.043216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.188918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.316963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.275420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.236084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.701187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.830492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.558203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.814075] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6330.817794] Lustre: Skipped 488 previous similar messages [ 6330.835160] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6330.845199] Lustre: Skipped 488 previous similar messages [ 6330.854084] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6330.863584] Lustre: Skipped 488 previous similar messages [ 6330.869084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.904982] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6330.907914] Lustre: Skipped 488 previous similar messages [ 6332.131609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.070844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.154156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.179696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.401963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.208492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.294577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.101854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.210050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.173446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.245502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.567195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.681863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.601604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.551236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.491970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.612682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.403319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.523706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.621040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.629360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.835096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.085270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.339857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.482464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.605423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.586596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.702993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.846488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.808878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.812876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.882831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.826831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.936402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.927509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.941369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.337943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.650968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.796409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.853169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.921307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.131314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.197960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.207925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.372544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.486974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.216616] LustreError: 164419:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6861.224706] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6861.237378] LustreError: 162840:0:(osp_precreate.c:652:osp_precreate_send()) Skipped 1 previous similar message [ 7592.935788] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7592.939565] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7592.949730] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7592.958363] Lustre: Skipped 47 previous similar messages [ 7592.967349] Lustre: Skipped 1 previous similar message [ 7596.102866] Lustre: server umount lustre-OST0000 complete [ 7603.174344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7606.411371] Lustre: server umount lustre-MDT0000 complete [ 7608.295091] LustreError: 163775:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7608.307575] LustreError: 163775:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7609.704674] LustreError: 162801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780341396 with bad export cookie 8975604283643920069 [ 7609.706755] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7609.733893] LustreError: 162801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7610.040741] Lustre: server umount lustre-MDT0001 complete [ 7620.819634] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 7623.295844] Key type lgssc unregistered [ 7623.531109] LNet: 217780:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7623.536099] LNetError: 217780:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7623.551793] LNet: Removed LNI 192.168.202.131@tcp [ 7624.464837] Key type .llcrypt unregistered [ 7624.470273] Key type ._llcrypt unregistered [ 7648.212746] Key type ._llcrypt registered [ 7648.215761] Key type .llcrypt registered [ 7648.320891] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 7649.289775] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7649.338484] alg: No test for adler32 (adler32-zlib) [ 7650.352404] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 7650.548846] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 7652.175483] Key type lgssc registered [ 7653.367822] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7660.400538] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7660.416958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7661.816258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7665.700960] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7671.265296] LustreError: 219183:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7671.286540] LustreError: 219183:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7674.070718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7674.372709] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7678.117582] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7683.116493] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7686.735310] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7692.357317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7692.552706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7697.781872] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7703.054586] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7706.928867] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7711.655751] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7713.074620] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7713.084750] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7713.092264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.101353] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7713.105479] LustreError: 219203:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7715.298509] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7715.311039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7715.333396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7717.347441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7717.348845] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7718.846144] Lustre: lustre-OST0000: Not available for connect from 192.168.202.31@tcp (stopping) [ 7718.852094] Lustre: Skipped 1 previous similar message [ 7721.207575] Lustre: server umount lustre-OST0000 complete [ 7733.148863] LustreError: 219203:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7753.612826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7753.869695] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7757.991096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7758.001319] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7759.721661] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7765.158227] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7766.858221] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7767.224818] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7767.235165] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7770.775259] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa02049907800.ost_server_uuid 50 [ 7772.288464] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa02049907800.ost_server_uuid in FULL state after 0 sec [ 7777.760723] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7777.766577] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7777.772113] Lustre: Skipped 1 previous similar message [ 7777.777541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7782.885024] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7782.891689] Lustre: Skipped 1 previous similar message [ 7783.095794] Lustre: server umount lustre-OST0000 complete [ 7786.982615] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7786.988311] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7793.120394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7793.132470] Lustre: Skipped 5 previous similar messages [ 7801.317508] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7801.496215] Lustre: server umount lustre-MDT0000 complete [ 7803.368714] LustreError: 219895:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7803.382508] LustreError: 219895:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7804.695815] LustreError: 222075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780341591 with bad export cookie 13957222530410366441 [ 7804.698571] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7804.708469] LustreError: 222075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7805.014522] Lustre: server umount lustre-MDT0001 complete [ 7815.347634] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 7818.077605] Key type lgssc unregistered [ 7818.345656] LNet: 223826:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7818.349180] LNetError: 223826:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7818.378608] LNet: Removed LNI 192.168.202.131@tcp [ 7819.208188] Key type .llcrypt unregistered [ 7819.209806] Key type ._llcrypt unregistered [ 7847.758344] Key type ._llcrypt registered [ 7847.764092] Key type .llcrypt registered [ 7847.879522] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 7849.089724] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7849.130796] alg: No test for adler32 (adler32-zlib) [ 7850.190899] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 7850.384038] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 7852.039212] Key type lgssc registered [ 7853.374106] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7862.314216] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7862.334993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7863.880844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7868.676863] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7874.017798] LustreError: 225537:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7874.038609] LustreError: 225537:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7877.750648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7878.152806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7882.364336] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7885.291532] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7891.977145] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7892.272421] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7898.572849] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7907.804436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7907.987192] Lustre: lustre-OST0001: new disk, initializing [ 7907.989935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7908.000720] Lustre: 227794:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7908.133335] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7909.185986] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7914.092373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7914.095727] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 7914.107273] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7914.171077] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7923.651325] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7929.001726] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7944.672124] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7944.678287] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7944.690199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7945.187993] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7945.198149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7945.209993] Lustre: Skipped 2 previous similar messages [ 7945.214250] Lustre: Skipped 2 previous similar messages [ 7949.875096] Lustre: server umount lustre-MDT0000 complete [ 7950.311084] LustreError: 225538:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7950.334714] LustreError: 225538:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7952.907615] LustreError: 227793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780341739 with bad export cookie 2533578374342673957 [ 7952.918106] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7952.919144] LustreError: 227793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7953.206377] Lustre: server umount lustre-MDT0001 complete [ 7966.865416] Lustre: server umount lustre-OST0000 complete [ 7980.357410] Lustre: server umount lustre-OST0001 complete [ 7999.250307] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8008.554149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8009.075780] LustreError: 230868:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8009.150932] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8012.814610] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8014.307591] LustreError: 230869:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8018.405663] LustreError: 230868:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8019.809039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8020.111143] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8023.766825] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8025.875589] Lustre: 231978:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8030.574190] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8033.893303] LustreError: 232331:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8033.918380] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8035.473909] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8042.031573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8042.187062] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8042.190928] Lustre: Skipped 1 previous similar message [ 8046.443634] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8052.301457] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8055.381838] Lustre: 233814:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8064.994835] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8064.999805] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8065.017651] Lustre: Skipped 2 previous similar messages [ 8065.036716] Lustre: Skipped 4 previous similar messages [ 8070.118305] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8070.624435] Lustre: server umount mylustre-MDT0000 complete [ 8073.474119] LustreError: 230843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780341860 with bad export cookie 2533578374342676764 [ 8073.475071] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8073.478987] LustreError: 230843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8073.781499] Lustre: server umount mylustre-MDT0001 complete [ 8087.170932] Lustre: server umount mylustre-OST0000 complete [ 8099.292993] Lustre: server umount mylustre-OST0001 complete [ 8115.982407] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8124.069309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8124.410078] LustreError: 236107:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8124.428560] LustreError: 236107:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8124.484646] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8127.750325] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8134.923691] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8138.197214] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8140.377301] Lustre: 237215:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8145.403972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8149.758515] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 8150.679500] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8157.122898] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8162.272720] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8168.490760] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8171.514156] Lustre: 239051:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8181.221776] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8181.230262] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8181.237782] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8181.243058] Lustre: Skipped 4 previous similar messages [ 8181.250895] Lustre: Skipped 6 previous similar messages [ 8182.722143] Lustre: server umount tfs-MDT0000 complete [ 8185.226945] LustreError: 236084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780341972 with bad export cookie 2533578374342678430 [ 8185.232436] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8185.477045] Lustre: server umount tfs-MDT0001 complete [ 8198.449630] Lustre: server umount tfs-OST0000 complete [ 8211.600556] Lustre: server umount tfs-OST0001 complete [ 8227.626185] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8234.371413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8234.739822] LustreError: 241285:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8234.759190] LustreError: 241285:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 8234.789708] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8234.792787] Lustre: Skipped 3 previous similar messages [ 8237.988788] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8244.175803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8248.172615] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8250.900613] Lustre: 242393:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8256.526977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8259.887922] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 8261.815489] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8268.382185] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8273.195185] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8279.241485] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8282.130221] Lustre: 244229:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8287.058852] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 15:27:52 (1780342072) [ 8301.024424] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8301.027099] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8301.032872] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8315.359138] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8315.490970] Lustre: server umount lustre-MDT0000 complete [ 8315.873196] LustreError: 241285:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8315.882424] LustreError: 241285:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 8317.888149] LustreError: 242011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342104 with bad export cookie 2533578374342679872 [ 8317.891980] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8317.906675] LustreError: 242011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 8320.996856] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8321.005187] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8321.005603] Lustre: Skipped 4 previous similar messages [ 8321.014058] Lustre: Skipped 9 previous similar messages [ 8332.256176] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8332.409809] Lustre: server umount lustre-MDT0001 complete [ 8341.327767] Lustre: server umount lustre-OST0000 complete [ 8350.406344] Lustre: server umount lustre-OST0001 complete [ 8355.429200] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 8362.198557] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8386.669683] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8386.669683] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8386.669683] [ 8386.684730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8386.832191] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8386.851159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8386.904494] Lustre: lustre-MDT0000: new disk, initializing [ 8386.968947] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8386.973015] Lustre: Skipped 3 previous similar messages [ 8386.984830] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8389.840547] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8397.802153] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8397.802153] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8397.802153] [ 8397.822579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8397.890785] Lustre: 247380:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8397.958361] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8397.962618] Lustre: Skipped 1 previous similar message [ 8398.017927] Lustre: lustre-MDT0001: new disk, initializing [ 8398.116480] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8398.129282] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8401.341156] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8409.712244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8409.877199] Lustre: lustre-OST0000: new disk, initializing [ 8409.880802] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8409.886743] Lustre: 248929:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8410.990175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8411.009231] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8411.109590] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8414.562208] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8421.878051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8422.000139] Lustre: lustre-OST0001: new disk, initializing [ 8422.005965] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8422.013980] Lustre: 249781:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8423.820509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8423.825571] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8423.931714] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8426.116575] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8433.378303] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 15:30:19 (1780342219) [ 8442.213512] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 15:30:27 (1780342227) [ 8457.189916] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8457.192188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8457.195395] LustreError: Skipped 1 previous similar message [ 8457.196594] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8457.196602] Lustre: Skipped 6 previous similar messages [ 8457.223138] Lustre: Skipped 1 previous similar message [ 8461.403283] Lustre: server umount lustre-OST0000 complete [ 8464.864623] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8464.866797] LustreError: 248914:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8464.878936] LustreError: 248914:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 8478.175305] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8478.322819] Lustre: server umount lustre-MDT0000 complete [ 8480.858133] LustreError: 248157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342267 with bad export cookie 2533578374342681202 [ 8480.861805] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8480.864376] LustreError: 248157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8487.110452] Lustre: server umount lustre-MDT0001 complete [ 8493.936796] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 8507.246135] Lustre: server umount lustre-OST0001 complete [ 8511.412776] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 8521.181157] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8543.025904] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8543.181971] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8543.205470] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8543.249695] Lustre: lustre-MDT0000: new disk, initializing [ 8543.317402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8545.694206] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8552.602166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8552.680902] Lustre: 254246:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8552.821025] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8555.469318] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8560.903659] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8563.118639] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8567.448578] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8567.611736] Lustre: lustre-OST0000: new disk, initializing [ 8567.614023] Lustre: Skipped 1 previous similar message [ 8567.616687] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8567.620046] Lustre: Skipped 2 previous similar messages [ 8567.624076] Lustre: 255989:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8569.246125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8569.254107] Lustre: Skipped 1 previous similar message [ 8569.262342] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8569.364393] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8571.068906] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8575.638528] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8579.557774] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8579.558168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8579.560985] LustreError: Skipped 1 previous similar message [ 8579.561917] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8579.574887] Lustre: Skipped 5 previous similar messages [ 8579.576267] Lustre: Skipped 13 previous similar messages [ 8585.001711] Lustre: server umount lustre-OST0000 complete [ 8593.366150] Lustre: server umount lustre-MDT0000 complete [ 8595.829566] LustreError: 254239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342382 with bad export cookie 2533578374342682714 [ 8595.841039] LustreError: 254239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8595.842703] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8596.055905] Lustre: server umount lustre-MDT0001 complete [ 8601.745670] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 8603.412561] Key type lgssc unregistered [ 8603.602800] LNet: 257619:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8603.607872] LNetError: 257619:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8603.621164] LNet: Removed LNI 192.168.202.131@tcp [ 8604.171152] Key type .llcrypt unregistered [ 8604.172980] Key type ._llcrypt unregistered [ 8619.316532] Key type ._llcrypt registered [ 8619.318937] Key type .llcrypt registered [ 8619.383863] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8620.069518] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8620.086669] alg: No test for adler32 (adler32-zlib) [ 8621.020591] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 8621.156995] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 8622.759407] Key type lgssc registered [ 8623.387827] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8627.951331] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8627.960434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8629.242226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8631.522789] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8637.366076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8637.646199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8640.405386] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8643.638222] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8645.723413] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8649.774618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8650.024157] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8653.442660] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8655.164545] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8656.836549] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8657.448267] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8660.462623] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8660.466334] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8660.490150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8662.571384] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8662.574685] Lustre: Skipped 1 previous similar message [ 8664.796810] Lustre: server umount lustre-OST0000 complete [ 8667.616594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8667.622263] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8667.625496] Lustre: Skipped 1 previous similar message [ 8672.736524] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8672.739677] Lustre: Skipped 2 previous similar messages [ 8673.452206] Lustre: server umount lustre-MDT0000 complete [ 8675.774146] LustreError: 259039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342462 with bad export cookie 9167639064926325216 [ 8675.777337] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8675.780093] LustreError: 259039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8676.018376] Lustre: server umount lustre-MDT0001 complete [ 8681.942555] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 8683.428582] Key type lgssc unregistered [ 8683.605491] LNet: 262285:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8683.609979] LNetError: 262285:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8683.620420] LNet: Removed LNI 192.168.202.131@tcp [ 8684.026636] Key type .llcrypt unregistered [ 8684.028765] Key type ._llcrypt unregistered [ 8694.428592] Key type ._llcrypt registered [ 8694.429817] Key type .llcrypt registered [ 8694.479859] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8695.038793] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8695.091496] alg: No test for adler32 (adler32-zlib) [ 8696.074641] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 8696.191219] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 8697.824140] Key type lgssc registered [ 8698.568764] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8703.627934] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8703.640096] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8719.135326] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8724.255399] LustreError: 263438:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.131@tcp: failed processing log, type 4: rc = -110 [ 8750.879224] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8754.277167] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8757.778455] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8759.225407] Lustre: server umount lustre-OST0000 complete [ 8768.764530] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 8770.130140] Key type lgssc unregistered [ 8770.296546] LNet: 264942:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8770.300435] LNetError: 264942:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8770.308728] LNet: Removed LNI 192.168.202.131@tcp [ 8770.666769] Key type .llcrypt unregistered [ 8770.669274] Key type ._llcrypt unregistered [ 8790.382951] Key type ._llcrypt registered [ 8790.384454] Key type .llcrypt registered [ 8790.434410] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8790.970438] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8791.046404] alg: No test for adler32 (adler32-zlib) [ 8791.938371] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 8792.076791] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 8793.679218] Key type lgssc registered [ 8794.328654] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8801.559435] loop0: detected capacity change from 0 to 204800000 [ 8801.648122] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8801.654395] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8801.792079] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8802.313260] loop0: detected capacity change from 0 to 204800000 [ 8802.429128] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8802.434359] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8802.597318] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8803.124452] loop0: detected capacity change from 0 to 204800000 [ 8803.216449] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8803.222361] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8803.282130] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8803.842831] loop0: detected capacity change from 0 to 204800000 [ 8803.923856] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8804.551376] loop0: detected capacity change from 0 to 204800000 [ 8804.585661] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8806.769584] loop0: detected capacity change from 0 to 204800000 [ 8806.794211] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8808.771640] loop0: detected capacity change from 0 to 204800000 [ 8808.796122] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8810.825636] loop0: detected capacity change from 0 to 204800000 [ 8810.845808] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8812.858824] loop0: detected capacity change from 0 to 204800000 [ 8812.937778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8812.956504] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8813.026273] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8813.040098] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8813.046503] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8813.057755] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8813.063992] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8813.072840] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8813.085274] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8813.092910] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8813.101445] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8813.108771] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8813.112972] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8813.117448] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8813.122312] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8813.126777] Lustre: 268312:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8816.876281] Lustre: server umount MGS complete [ 8817.479601] loop0: detected capacity change from 0 to 204800000 [ 8817.521099] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8817.822075] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8817.857229] Lustre: lustre-MDT0000: Aborting client recovery [ 8818.414395] loop1: detected capacity change from 0 to 204800000 [ 8818.506687] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8818.588150] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8818.602634] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8818.606776] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8818.611490] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8818.616761] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8818.621363] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8818.626443] Lustre: 268745:0:(osd_scrub.c:1840:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8818.639394] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8818.748410] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8818.775587] Lustre: lustre-MDT0001: Aborting client recovery [ 8819.290802] loop2: detected capacity change from 0 to 204800000 [ 8819.359816] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8819.398038] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8819.404669] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8819.411088] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8819.416723] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8819.423140] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8819.429093] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8819.434289] Lustre: 268826:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8819.501720] Lustre: Found index 0 for lustre-OST0000, updating log [ 8819.530881] Lustre: 268844:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8819.584851] Lustre: lustre-OST0000: Aborting recovery [ 8820.184692] loop3: detected capacity change from 0 to 204800000 [ 8820.245885] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8820.285436] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8820.290796] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8820.294411] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8820.298698] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8820.302916] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8820.307555] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8820.313189] Lustre: 268910:0:(osd_scrub.c:1840:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8820.352274] Lustre: 268914:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8820.398057] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8820.403103] Lustre: Skipped 1 previous similar message [ 8820.416884] Lustre: lustre-OST0001: Aborting recovery [ 8825.843888] LustreError: 268688:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8827.877599] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8827.878669] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8827.899676] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8827.908815] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8828.905747] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8828.907461] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8831.887622] LustreError: 268655:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 485us: rc = -2 [ 8844.258289] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8844.264049] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8844.264185] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8844.264221] Lustre: Skipped 1 previous similar message [ 8844.268886] Lustre: Skipped 2 previous similar messages [ 8849.378480] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8849.383921] Lustre: Skipped 5 previous similar messages [ 8849.608077] Lustre: server umount lustre-MDT0000 complete [ 8850.085824] LustreError: 268637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342636 with bad export cookie 8724353496348602694 [ 8850.098604] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8850.102392] LustreError: 268637:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8854.495730] LustreError: 268652:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8854.495893] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8854.497492] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8854.504724] LustreError: 268652:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8854.514168] Lustre: Skipped 2 previous similar messages [ 8855.519888] LustreError: 268653:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8855.529082] LustreError: 268653:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8856.407485] Lustre: server umount lustre-MDT0001 complete [ 8871.392301] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8871.468552] Lustre: server umount lustre-OST0000 complete [ 8878.231425] Lustre: server umount lustre-OST0001 complete [ 8881.860564] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 15:37:47 (1780342667) [ 8895.213515] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 8900.038696] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 8916.502515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8916.607645] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8916.623845] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8916.665325] Lustre: lustre-MDT0000: new disk, initializing [ 8916.726456] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8916.738717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8918.628186] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8923.992716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8924.040686] Lustre: 272941:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8924.071313] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8924.073929] Lustre: Skipped 1 previous similar message [ 8924.114634] Lustre: lustre-MDT0001: new disk, initializing [ 8924.156753] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8924.180651] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8924.188598] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8925.941453] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8930.191751] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8932.181245] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8935.571130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8935.688935] Lustre: lustre-OST0000: new disk, initializing [ 8935.692319] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8935.696626] Lustre: 274682:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8935.740633] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8939.104239] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 8943.955604] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8945.201681] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8945.206386] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8945.266427] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8952.841173] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8960.638542] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8966.341435] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8974.362103] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9012.192325] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9012.197544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9012.204939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9012.207066] Lustre: Skipped 3 previous similar messages [ 9016.802569] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9016.803049] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9016.810507] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9026.015177] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9026.127244] Lustre: server umount lustre-OST0000 complete [ 9031.649053] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9031.655989] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9031.667460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9031.672637] Lustre: Skipped 3 previous similar messages [ 9034.556679] Lustre: server umount lustre-MDT0000 complete [ 9036.671768] LustreError: 272931:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342823 with bad export cookie 8724353496348605438 [ 9036.672690] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9036.676879] LustreError: 272931:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 9036.869753] Lustre: server umount lustre-MDT0001 complete [ 9041.452825] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9043.263536] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9045.995118] Lustre: server umount MGS complete [ 9057.624728] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9057.879037] LustreError: 278470:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9057.900224] LustreError: 278483:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9057.937896] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9063.403515] LustreError: 278471:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9063.900595] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9067.500324] LustreError: 278470:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9074.852114] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9079.838770] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9087.129915] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9091.696761] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9099.442374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9099.752457] LustreError: 280070:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9099.781993] LustreError: 280070:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9100.008364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9100.011657] Lustre: Skipped 1 previous similar message [ 9108.192435] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9116.403338] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9144.296260] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9144.296493] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9144.300643] LustreError: Skipped 1 previous similar message [ 9144.301382] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9144.313454] Lustre: Skipped 2 previous similar messages [ 9144.326114] Lustre: Skipped 3 previous similar messages [ 9158.111519] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9158.238857] Lustre: server umount lustre-OST0000 complete [ 9162.209805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9162.218855] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9166.914314] Lustre: server umount lustre-MDT0000 complete [ 9169.562359] LustreError: 280971:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780342956 with bad export cookie 8724353496348607405 [ 9169.568973] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9169.856180] Lustre: server umount lustre-MDT0001 complete [ 9178.855074] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 9180.929433] Key type lgssc unregistered [ 9181.150978] LNet: 282062:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9181.157675] LNetError: 282062:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9181.166837] LNet: Removed LNI 192.168.202.131@tcp [ 9181.682740] Key type .llcrypt unregistered [ 9181.685121] Key type ._llcrypt unregistered [ 9211.046515] Key type ._llcrypt registered [ 9211.048450] Key type .llcrypt registered [ 9211.112575] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 9224.322660] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 9225.031086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9225.043183] alg: No test for adler32 (adler32-zlib) [ 9226.023326] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 9226.201496] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 9227.839191] Key type lgssc registered [ 9228.822822] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9255.021670] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9255.038664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9256.231112] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9256.255333] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9256.332916] Lustre: lustre-MDT0000: new disk, initializing [ 9256.383044] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9256.408768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9259.376710] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9267.706924] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9267.816433] Lustre: 285985:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9267.860976] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9267.864642] Lustre: Skipped 1 previous similar message [ 9267.911922] Lustre: lustre-MDT0001: new disk, initializing [ 9267.971139] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9267.996968] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9268.013508] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9271.201412] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9277.172167] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9280.259209] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9285.560487] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9285.791785] Lustre: lustre-OST0000: new disk, initializing [ 9285.795756] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9285.802363] Lustre: 287727:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9285.890268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9290.703804] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9295.897134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9295.905237] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9296.020077] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9296.318506] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9302.697747] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9308.622464] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9315.291313] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9322.716063] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9362.404610] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9362.405636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9362.421306] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9362.433520] Lustre: Skipped 1 previous similar message [ 9365.566614] Lustre: server umount lustre-OST0000 complete [ 9370.594456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9370.599222] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9370.608552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9370.613996] Lustre: Skipped 1 previous similar message [ 9372.641251] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9372.644569] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9372.659798] Lustre: Skipped 1 previous similar message [ 9374.886821] Lustre: server umount lustre-MDT0000 complete [ 9377.789897] LustreError: 285976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780343164 with bad export cookie 2494920867124764544 [ 9377.800479] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9377.806593] LustreError: 285976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9378.122831] Lustre: server umount lustre-MDT0001 complete [ 9384.672780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9388.092889] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9391.806269] Lustre: server umount MGS complete [ 9400.566452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9400.911632] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9400.924381] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9400.936346] LustreError: 291420:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9400.992475] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9404.112882] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9406.439784] LustreError: 291408:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9410.531273] LustreError: 291407:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9411.498643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9411.679956] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9414.604486] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9419.474810] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9423.702111] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9430.137590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9430.394823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9435.749531] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9440.463685] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9472.480739] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9472.488441] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9472.499386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9476.579352] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9476.580470] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9476.581745] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9476.587217] Lustre: Skipped 1 previous similar message [ 9478.513784] Lustre: server umount lustre-OST0000 complete [ 9486.817160] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9486.822459] Lustre: Skipped 3 previous similar messages [ 9487.277631] Lustre: server umount lustre-MDT0000 complete [ 9489.886617] LustreError: 291389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780343276 with bad export cookie 2494920867124766434 [ 9489.893676] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9489.898345] LustreError: 291389:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9490.135826] Lustre: server umount lustre-MDT0001 complete [ 9497.441408] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [ 9499.181329] Key type lgssc unregistered [ 9499.338933] LNet: 294994:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9499.345201] LNetError: 294994:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9499.359648] LNet: Removed LNI 192.168.202.131@tcp [ 9499.902714] Key type .llcrypt unregistered [ 9499.904699] Key type ._llcrypt unregistered [ 9514.193200] Key type ._llcrypt registered [ 9514.194768] Key type .llcrypt registered [ 9514.277309] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [ 9523.651027] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 9524.504462] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9524.539655] alg: No test for adler32 (adler32-zlib) [ 9525.515161] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [ 9525.663115] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [ 9527.311269] Key type lgssc registered [ 9528.165638] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9543.439455] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [ 9555.184435] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9555.195562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9556.410846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9556.435947] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9556.489614] Lustre: lustre-MDT0000: new disk, initializing [ 9556.560734] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9556.572989] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9559.476515] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [ 9567.725116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9568.065596] Lustre: lustre-OST0000: new disk, initializing [ 9568.070320] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9568.076697] Lustre: Skipped 1 previous similar message [ 9568.082878] Lustre: 298954:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9568.161064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9571.244108] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9571.253761] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9571.276868] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9572.916341] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10147.878093] Lustre: server umount lustre-MDT0000 complete [10150.028961] LustreError: 298157:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780343936 with bad export cookie 16093861366542448774 [10150.035760] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10159.698690] Lustre: server umount lustre-OST0000 complete [10175.644479] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [10181.586375] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10204.231834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10204.356246] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10204.370664] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10204.407311] Lustre: lustre-MDT0000: new disk, initializing [10204.441378] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10204.449510] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10207.115453] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10214.480063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10214.568876] Lustre: 303191:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10214.617521] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10214.619926] Lustre: Skipped 1 previous similar message [10214.681199] Lustre: lustre-MDT0001: new disk, initializing [10214.742127] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10214.761308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10214.766485] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10217.141639] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10221.671810] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10223.365219] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10226.959465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10227.131387] Lustre: lustre-OST0000: new disk, initializing [10227.133601] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10227.137350] Lustre: 304933:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10227.200807] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10230.504838] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10231.821866] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10231.826511] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10231.876136] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10235.351359] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10237.815642] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10237.934687] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10240.228602] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10240.354453] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10242.020267] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10242.023859] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10242.026655] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10242.039134] Lustre: Skipped 1 previous similar message [10242.046841] Lustre: Skipped 1 previous similar message [10247.148600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10247.155066] Lustre: Skipped 1 previous similar message [10247.658371] Lustre: server umount lustre-OST0000 complete [10250.720775] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10250.725363] LustreError: Skipped 1 previous similar message [10250.727653] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10250.740513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10252.259444] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10255.737642] Lustre: server umount lustre-MDT0000 complete [10257.599558] LustreError: 303979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344044 with bad export cookie 16093861366545389579 [10257.599836] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10257.606726] LustreError: 303979:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10257.787530] Lustre: server umount lustre-MDT0001 complete [10259.728810] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10260.665785] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 16:00:46 (1780344046) [10264.358077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10264.600133] LustreError: 307093:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10264.606531] LustreError: 307093:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10264.647392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10266.651656] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10269.666773] LustreError: 307094:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10271.529521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10271.748570] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10274.363672] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10277.847197] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10279.932668] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10283.568100] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10283.740132] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10286.728330] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10290.030888] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10293.752918] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10293.874754] Lustre: lustre-OST0001: new disk, initializing [10293.878388] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10293.885054] Lustre: 309597:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10296.958247] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10302.240718] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10302.327096] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10304.015443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10304.023340] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10304.057115] LustreError: 310459:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10304.062086] LustreError: 310459:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10304.282781] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa02043c8e000.ost_server_uuid 50 [10305.127032] LustreError: 310459:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10305.161265] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa02043c8e000.ost_server_uuid in FULL state after 0 sec [10306.144880] LustreError: 310455:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10306.161955] LustreError: 310455:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10307.396775] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa02043c8e000.ost_server_uuid 50 [10308.195158] LustreError: 310455:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10308.200042] LustreError: 310455:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10308.361730] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa02043c8e000.ost_server_uuid in FULL state after 0 sec [10309.247235] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10324.455119] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10324.459487] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10324.463747] Lustre: Skipped 1 previous similar message [10324.469167] Lustre: Skipped 2 previous similar messages [10327.185556] Lustre: server umount lustre-OST0001 complete [10329.572229] LustreError: 308697:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10329.583366] LustreError: 308697:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10331.105393] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10331.112153] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10331.118684] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10331.121337] Lustre: Skipped 2 previous similar messages [10333.158162] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10334.692366] LustreError: 310461:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10334.705882] LustreError: 310461:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10336.049559] Lustre: server umount lustre-OST0000 complete [10338.273747] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10339.808595] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10339.809734] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10339.816536] Lustre: Skipped 2 previous similar messages [10339.819758] Lustre: Skipped 4 previous similar messages [10344.314406] Lustre: server umount lustre-MDT0000 complete [10344.932807] LustreError: 307093:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10346.520939] LustreError: 307075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344133 with bad export cookie 16093861366545390503 [10346.520939] LustreError: 307076:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344133 with bad export cookie 16093861366545390503 [10346.523964] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10346.528137] LustreError: 307075:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10346.757137] Lustre: server umount lustre-MDT0001 complete [10352.318228] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [10353.922957] Key type lgssc unregistered [10354.084245] LNet: 312037:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10354.094469] LNetError: 312037:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10354.104941] LNet: Removed LNI 192.168.202.131@tcp [10354.533653] Key type .llcrypt unregistered [10354.537838] Key type ._llcrypt unregistered [10366.920947] Key type ._llcrypt registered [10366.922827] Key type .llcrypt registered [10367.007382] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10367.662394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10367.675139] alg: No test for adler32 (adler32-zlib) [10368.598314] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [10368.743526] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [10370.375208] Key type lgssc registered [10371.018539] Lustre: Echo OBD driver; http://www.lustre.org/ [10375.138835] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10375.153780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10376.425090] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10378.764680] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10381.671432] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10385.415490] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10385.640678] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10387.830231] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10392.635198] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10392.781773] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10395.443448] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10395.880149] LustreError: 314965:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10395.886399] LustreError: 314965:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10398.127639] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10401.251347] LustreError: 314967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10401.479693] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10401.568975] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10402.603733] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10402.607110] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10404.251855] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10406.544161] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [10410.400170] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10410.504731] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10494.945311] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10494.948459] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10494.953033] Lustre: Skipped 1 previous similar message [10494.953416] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10494.957398] Lustre: Skipped 1 previous similar message [10498.562276] Lustre: server umount lustre-OST0001 complete [10500.067493] LustreError: 315984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10500.073338] LustreError: 315984:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10503.648257] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10503.651375] LustreError: Skipped 1 previous similar message [10503.653152] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10503.659089] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10505.184749] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10505.186816] LustreError: 314967:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.187263] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10505.187881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10505.192522] LustreError: 314967:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10506.445607] Lustre: server umount lustre-OST0000 complete [10508.767881] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10508.771494] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10508.778142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10508.781346] Lustre: Skipped 1 previous similar message [10510.065097] Lustre: server umount lustre-MDT0000 complete [10510.306641] LustreError: 313417:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10510.313239] LustreError: 313417:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10511.478099] LustreError: 316752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344298 with bad export cookie 2230726261765734484 [10511.481935] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10511.482799] LustreError: 316752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10511.617245] Lustre: server umount lustre-MDT0001 complete [10515.634341] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [10516.876480] Key type lgssc unregistered [10517.076924] LNet: 317965:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10517.083279] LNetError: 317965:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10517.099687] LNet: Removed LNI 192.168.202.131@tcp [10517.498225] Key type .llcrypt unregistered [10517.499775] Key type ._llcrypt unregistered [10539.277746] Key type ._llcrypt registered [10539.279068] Key type .llcrypt registered [10539.322342] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10539.768576] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10539.821152] alg: No test for adler32 (adler32-zlib) [10540.699620] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [10540.809579] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [10542.407364] Key type lgssc registered [10543.000599] Lustre: Echo OBD driver; http://www.lustre.org/ [10547.328125] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10547.342104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10548.619054] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10550.299498] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10554.569912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10554.758910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10556.468994] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10557.958611] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10560.939334] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10561.079131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10563.421933] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10564.133533] LustreError: 321667:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10564.141139] LustreError: 321667:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10566.648690] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10566.753526] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10569.179299] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10573.802229] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10573.802233] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10577.776814] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10579.574723] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10584.035638] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10584.040777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10584.044320] Lustre: Skipped 2 previous similar messages [10584.047603] Lustre: Skipped 3 previous similar messages [10589.152350] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10589.155019] Lustre: Skipped 3 previous similar messages [10589.928675] Lustre: server umount lustre-MDT0000 complete [10591.417025] LustreError: 320933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344378 with bad export cookie 13594820417828538624 [10591.417672] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10591.420659] LustreError: 320933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10591.556753] Lustre: server umount lustre-MDT0001 complete [10602.510032] Lustre: server umount lustre-OST0000 complete [10613.302745] Lustre: server umount lustre-OST0001 complete [10620.337746] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10624.518196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10624.721201] LustreError: 325388:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10624.774349] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10626.481302] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10630.085975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10630.112302] LustreError: 325389:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10631.792017] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10632.967690] Lustre: 326496:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10635.547063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10635.671055] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10635.675740] Lustre: Skipped 1 previous similar message [10637.763683] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10640.805583] LustreError: 326853:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10641.125404] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10643.210964] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10649.583883] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10649.583891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10657.354469] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10658.898619] Lustre: 328353:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10660.832080] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10660.834647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10660.841513] Lustre: Skipped 1 previous similar message [10660.844178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10664.931987] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10664.933023] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10664.940449] Lustre: Skipped 2 previous similar messages [10664.946938] Lustre: Skipped 3 previous similar messages [10666.460381] Lustre: server umount lustre-MDT0000 complete [10667.840740] LustreError: 325369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344454 with bad export cookie 13594820417828540297 [10667.843762] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10667.844956] LustreError: 325369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10667.976637] Lustre: server umount lustre-MDT0001 complete [10679.241791] Lustre: server umount lustre-OST0000 complete [10691.191612] Lustre: server umount lustre-OST0001 complete [10692.767607] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10693.410562] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10694.132826] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 16:08:00 (1780344480) [10703.438200] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10707.354787] loop0: detected capacity change from 0 to 1099511627776 [10707.521765] SGI XFS with ACLs, security attributes, quota, no debug enabled [10707.527423] *** VALIDATE xfs *** [10707.536141] XFS (loop0): Mounting V5 Filesystem [10707.551456] XFS (loop0): Ending clean mount [10708.031422] loop1: detected capacity change from 0 to 18691697672192 [10730.489380] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [10734.053641] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10747.800595] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10747.898086] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10747.911718] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10747.949792] Lustre: lustre-MDT0000: new disk, initializing [10747.978884] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10747.982016] Lustre: Skipped 1 previous similar message [10747.988459] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10749.498912] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10754.266910] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10754.314281] Lustre: 333414:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10754.336888] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10754.338540] Lustre: Skipped 1 previous similar message [10754.375408] Lustre: lustre-MDT0001: new disk, initializing [10754.435128] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10754.439531] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10756.022657] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10759.418872] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10760.940071] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10763.960691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10764.076800] Lustre: lustre-OST0000: new disk, initializing [10764.079719] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10764.083716] Lustre: 335159:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10765.886580] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10765.890029] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10765.932467] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10766.474753] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10770.594782] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10772.810434] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10772.906035] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10774.396764] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10774.483867] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10776.034496] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10776.034842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10776.035647] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10776.046064] Lustre: Skipped 1 previous similar message [10781.394386] Lustre: server umount lustre-OST0000 complete [10785.247791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10785.251516] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10785.257312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10785.259447] Lustre: Skipped 3 previous similar messages [10789.022082] Lustre: server umount lustre-MDT0000 complete [10790.440371] LustreError: 333408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344577 with bad export cookie 13594820417828541830 [10790.443131] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10790.445410] LustreError: 333408:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10790.565563] Lustre: server umount lustre-MDT0001 complete [10793.489578] XFS (loop0): Unmounting Filesystem [10795.240239] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 16:09:41 (1780344581) [10797.771495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10797.951995] LustreError: 337567:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10797.984400] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10797.986962] Lustre: Skipped 2 previous similar messages [10799.433493] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10802.895549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10804.500062] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10806.691892] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10808.156120] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10810.855513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10813.160069] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10815.297803] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10832.866654] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10832.868490] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10832.870534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10832.874564] Lustre: Skipped 2 previous similar messages [10832.876901] Lustre: Skipped 2 previous similar messages [10835.364722] Lustre: server umount lustre-OST0000 complete [10851.295107] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10851.388894] Lustre: server umount lustre-MDT0000 complete [10852.321839] LustreError: 337567:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10852.340343] LustreError: 337567:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10853.436429] LustreError: 337547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780344640 with bad export cookie 13594820417828542754 [10853.437356] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10853.444840] LustreError: 337547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10853.668575] Lustre: server umount lustre-MDT0001 complete [10858.780200] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [10860.054712] Key type lgssc unregistered [10860.208660] LNet: 340863:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10860.212450] LNetError: 340863:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10860.220564] LNet: Removed LNI 192.168.202.131@tcp [10860.579189] Key type .llcrypt unregistered [10860.580367] Key type ._llcrypt unregistered [10873.698840] Key type ._llcrypt registered [10873.700149] Key type .llcrypt registered [10873.742497] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [10874.129283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10874.180347] alg: No test for adler32 (adler32-zlib) [10875.048409] Lustre: Lustre: Build Version: 2.17.53_35_g98ac157 [10875.147334] LNet: Added LNI 192.168.202.131@tcp [8/256/0/180] [10876.743149] Key type lgssc registered [10877.139025] Lustre: Echo OBD driver; http://www.lustre.org/ [10880.075659] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10880.086410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10881.263183] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10882.814641] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10886.083701] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10886.234947] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10887.774772] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10889.955753] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10891.239745] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10893.542696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10893.656469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10895.629609] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10897.473636] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10911.712601] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10911.715716] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10911.720568] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [10913.195228] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.31@tcp (stopping) [10913.252757] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10916.978778] Lustre: server umount lustre-MDT0001 complete [10918.298696] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10918.305293] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10920.972649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10921.014643] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10921.017415] Lustre: Found index 1 for lustre-MDT0001, updating log [10921.032034] Lustre: 342378:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10921.160133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10922.634777] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [10926.564203] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10927.078424] LustreError: 345723:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10927.082444] LustreError: 345723:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10927.087146] LustreError: 342381:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10927.112968] LustreError: 345727:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.131@tcp: cfg command failed: rc = -17 [10927.117772] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [10927.117772] [11225.507639] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.31@tcp (stopping) [11225.514233] Lustre: Skipped 2 previous similar messages [11228.642086] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11228.643175] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11228.646124] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11228.650841] Lustre: Skipped 1 previous similar message [11233.760533] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11233.763250] Lustre: Skipped 3 previous similar messages [11239.391168] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11239.497039] Lustre: server umount lustre-MDT0001 complete [11240.859225] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.31@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11240.869678] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11244.003111] LustreError: 342385:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11244.011026] LustreError: 342385:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11244.326522] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11244.396061] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11244.398870] Lustre: Found index 1 for lustre-MDT0001, updating log [11244.416818] Lustre: 342378:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11244.573019] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11246.201932] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [11249.637482] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11252.199649] LustreError: 357574:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11252.202709] LustreError: 357574:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11252.206806] LustreError: 342381:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11252.209576] LustreError: 342381:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11252.228954] LustreError: 357577:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11252.231858] LustreError: 357577:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11252.233895] LustreError: 357577:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.131@tcp: cfg command failed: rc = -17 [11252.237450] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.131@tcp [11252.237450] [11551.715288] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11551.715789] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11551.723336] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11551.729548] Lustre: Skipped 4 previous similar messages [11554.867970] Lustre: server umount lustre-MDT0001 complete [11556.834602] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11556.841369] LustreError: 342384:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11559.126222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11559.160591] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11559.163585] Lustre: Found index 1 for lustre-MDT0001, updating log [11559.179170] Lustre: 342379:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11559.292255] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11560.938050] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [11564.519130] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11567.597233] LustreError: 369429:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11567.600359] LustreError: 369429:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11567.608736] LustreError: 342381:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11567.612760] LustreError: 342381:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11567.635057] LustreError: 369432:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11567.642873] LustreError: 369432:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.131@tcp: cfg command failed: rc = -114 [11567.647186] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11567.647186] [11881.954336] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11881.955651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11881.955689] Lustre: Skipped 1 previous similar message [11881.962436] Lustre: Skipped 3 previous similar messages [11881.964502] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11883.879087] Lustre: server umount lustre-MDT0000 complete [11885.659390] LustreError: 345255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780345672 with bad export cookie 5962438963268611629 [11885.660820] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11885.665867] LustreError: 345255:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11887.072996] LustreError: 343764:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11887.079011] LustreError: 343764:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11891.836363] Lustre: server umount lustre-MDT0001 complete [11893.580193] Lustre: server umount lustre-OST0000 complete [11897.494456] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [11901.250645] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [11941.474190] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 16:28:45 (1780345725) [11949.189361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11949.460793] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11949.536858] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11949.617369] Lustre: lustre-MDT0000: new disk, initializing [11949.795049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11949.813167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11954.384727] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [11965.247542] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11965.328253] Lustre: 383366:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11965.399451] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11965.402486] Lustre: Skipped 1 previous similar message [11965.512836] Lustre: lustre-MDT0001: new disk, initializing [11965.629845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11965.661264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11965.678291] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11970.948192] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [11979.035228] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11984.287833] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11993.254817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11993.597580] Lustre: lustre-OST0000: new disk, initializing [11993.603609] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11993.611327] Lustre: 385110:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11993.701348] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12000.969051] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_default_debug -1 all [12002.411524] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12002.424462] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12002.622564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12008.940390] Lustre: DEBUG MARKER: oleg231-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12017.125778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12017.136548] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12017.150901] Lustre: Skipped 1 previous similar message [12017.175620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12017.182796] Lustre: Skipped 3 previous similar messages [12021.366892] Lustre: server umount lustre-MDT0000 complete [12023.271331] LustreError: 384910:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12023.287914] LustreError: 384910:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12025.579224] LustreError: 383359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1780345812 with bad export cookie 5962438963268613568 [12025.613302] LustreError: MGC192.168.202.131@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12026.223928] Lustre: server umount lustre-MDT0001 complete [12039.577722] Lustre: server umount lustre-OST0000 complete [12063.417609] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing set_hostid [12072.250700] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing load_modules_local [12098.171279] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11761 sec ========= 16:31:23 (1780345883) [12099.420117] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 16:31:24 (1780345884) === [12102.352575] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 16:31:27 (1780345887) === [12131.179375] Lustre: DEBUG MARKER: oleg231-server.virtnet: executing unload_modules_local [12133.032939] Key type lgssc unregistered [12133.228689] LNet: 391310:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12133.235530] LNetError: 391310:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12133.250542] LNet: Removed LNI 192.168.202.131@tcp [12133.816980] Key type .llcrypt unregistered [12133.818375] Key type ._llcrypt unregistered