[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 489175537 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001013] APIC: Switch to symmetric I/O mode setup [ 0.002230] x2apic enabled [ 0.003010] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007025] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008016] pid_max: default: 32768 minimum: 301 [ 0.009152] LSM: Security Framework initializing [ 0.011019] Yama: becoming mindful. [ 0.012043] SELinux: Initializing. [ 0.013075] *** VALIDATE selinux *** [ 0.021462] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026302] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028091] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029126] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030136] *** VALIDATE tmpfs *** [ 0.032454] *** VALIDATE proc *** [ 0.033261] *** VALIDATE cgroup *** [ 0.034010] *** VALIDATE cgroup2 *** [ 0.035291] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037147] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039038] Spectre V2 : User space: Vulnerable [ 0.040011] Speculative Store Bypass: Vulnerable [ 0.043561] debug: unmapping init [mem 0xffffffffb8059000-0xffffffffb8060fff] [ 0.046212] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047751] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048028] ... version: 2 [ 0.049021] ... bit width: 48 [ 0.050014] ... generic registers: 4 [ 0.051023] ... value mask: 0000ffffffffffff [ 0.052020] ... max period: 00007fffffffffff [ 0.053020] ... fixed-purpose events: 3 [ 0.054017] ... event mask: 000000070000000f [ 0.055343] rcu: Hierarchical SRCU implementation. [ 0.057541] smp: Bringing up secondary CPUs ... [ 0.058619] x86: Booting SMP configuration: [ 0.059030] .... node #0, CPUs: #1 #2 #3 [ 0.068074] smp: Brought up 1 node, 4 CPUs [ 0.070015] smpboot: Max logical packages: 1 [ 0.071017] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.154390] node 0 deferred pages initialised in 81ms [ 0.158011] devtmpfs: initialized [ 0.159246] x86/mm: Memory block size: 128MB [ 0.161808] gcov: version magic: 0x41383552 [ 0.163437] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.167096] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.170591] pinctrl core: initialized pinctrl subsystem [ 0.173272] [ 0.174013] ************************************************************* [ 0.177014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.179013] ** ** [ 0.181013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.184015] ** ** [ 0.186012] ** This means that this kernel is built to expose internal ** [ 0.189021] ** IOMMU data structures, which may compromise security on ** [ 0.191012] ** your system. ** [ 0.193016] ** ** [ 0.196017] ** If you see this message and you are not debugging the ** [ 0.198013] ** kernel, report this immediately to your vendor! ** [ 0.200013] ** ** [ 0.203018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.205022] ************************************************************* [ 0.208150] NET: Registered protocol family 16 [ 0.210528] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.213083] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.216064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.219113] cpuidle: using governor menu [ 0.220825] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.223467] PCI: Using configuration type 1 for base access [ 0.225123] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.233104] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.234029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.237116] cryptd: max_cpu_qlen set to 1000 [ 0.239219] ACPI: Added _OSI(Module Device) [ 0.241122] ACPI: Added _OSI(Processor Device) [ 0.242011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.244013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.249097] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.256374] ACPI: Interpreter enabled [ 0.257054] ACPI: PM: (supports S0 S3 S4 S5) [ 0.258020] ACPI: Using IOAPIC for interrupt routing [ 0.260134] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.263448] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.270826] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.273051] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.276018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.279079] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.283258] acpiphp: Slot [2] registered [ 0.285184] acpiphp: Slot [5] registered [ 0.286127] acpiphp: Slot [6] registered [ 0.288148] acpiphp: Slot [7] registered [ 0.289122] acpiphp: Slot [8] registered [ 0.291115] acpiphp: Slot [9] registered [ 0.292126] acpiphp: Slot [10] registered [ 0.293138] acpiphp: Slot [3] registered [ 0.295132] acpiphp: Slot [4] registered [ 0.296094] acpiphp: Slot [11] registered [ 0.298130] acpiphp: Slot [12] registered [ 0.300148] acpiphp: Slot [13] registered [ 0.301110] acpiphp: Slot [14] registered [ 0.302074] acpiphp: Slot [15] registered [ 0.303031] acpiphp: Slot [16] registered [ 0.303990] acpiphp: Slot [17] registered [ 0.304080] acpiphp: Slot [18] registered [ 0.304977] acpiphp: Slot [19] registered [ 0.306087] acpiphp: Slot [20] registered [ 0.307089] acpiphp: Slot [21] registered [ 0.307937] acpiphp: Slot [22] registered [ 0.309056] acpiphp: Slot [23] registered [ 0.309951] acpiphp: Slot [24] registered [ 0.311109] acpiphp: Slot [25] registered [ 0.313103] acpiphp: Slot [26] registered [ 0.314123] acpiphp: Slot [27] registered [ 0.315096] acpiphp: Slot [28] registered [ 0.317105] acpiphp: Slot [29] registered [ 0.318114] acpiphp: Slot [30] registered [ 0.320101] acpiphp: Slot [31] registered [ 0.321082] PCI host bridge to bus 0000:00 [ 0.323025] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.325021] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.328030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.330024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.333024] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.335023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.337187] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.340024] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.343238] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.353671] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.357670] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.360015] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.361016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.363016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.366075] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.368854] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.371040] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.374779] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.380018] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.392021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.398020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.404036] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.414022] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.421022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.442023] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.454266] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.465016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.478018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.499028] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.509405] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.518018] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.527023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.552018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.562379] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.569017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.578017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.595017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.605937] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.614029] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.620017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.639020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.647190] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.660025] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.671025] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.691031] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.702986] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.706391] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.708364] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.710249] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.711170] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.714321] iommu: Default domain type: Passthrough [ 0.716443] SCSI subsystem initialized [ 0.717000] ACPI: bus type USB registered [ 0.719128] usbcore: registered new interface driver usbfs [ 0.720047] usbcore: registered new interface driver hub [ 0.721054] usbcore: registered new device driver usb [ 0.722096] pps_core: LinuxPPS API ver. 1 registered [ 0.723013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.725063] PTP clock support registered [ 0.727210] EDAC MC: Ver: 3.0.0 [ 0.728364] PCI: Using ACPI for IRQ routing [ 0.730484] NetLabel: Initializing [ 0.732017] NetLabel: domain hash size = 128 [ 0.732871] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.734067] NetLabel: unlabeled traffic allowed by default [ 0.735152] vgaarb: loaded [ 0.737047] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.738012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.742457] clocksource: Switched to clocksource kvm-clock [ 0.841569] VFS: Disk quotas dquot_6.6.0 [ 0.843755] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.846447] *** VALIDATE ramfs *** [ 0.847683] *** VALIDATE hugetlbfs *** [ 0.849246] pnp: PnP ACPI init [ 0.851138] pnp: PnP ACPI: found 6 devices [ 0.870657] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.873739] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.875713] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.877971] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.880589] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.883078] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.885849] NET: Registered protocol family 2 [ 0.888381] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.893557] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.897810] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.903806] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.907913] TCP: Hash tables configured (established 65536 bind 65536) [ 0.911127] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.916572] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.919274] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.921749] NET: Registered protocol family 1 [ 0.924163] RPC: Registered named UNIX socket transport module. [ 0.926109] RPC: Registered udp transport module. [ 0.927284] RPC: Registered tcp transport module. [ 0.928576] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.930259] NET: Registered protocol family 44 [ 0.931802] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.933862] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.936065] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.938437] PCI: CLS 0 bytes, default 64 [ 0.940086] Unpacking initramfs... [ 2.375639] debug: unmapping init [mem 0xffff9fdabcc54000-0xffff9fdabffbffff] [ 2.382530] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.385203] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.389958] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.957577] Initialise system trusted keyrings [ 2.959503] Key type blacklist registered [ 2.963201] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.972318] zbud: loaded [ 2.976714] *** VALIDATE nfs *** [ 2.978103] *** VALIDATE nfs4 *** [ 2.979715] pstore: using deflate compression [ 2.982943] Platform Keyring initialized [ 3.105672] NET: Registered protocol family 38 [ 3.108710] Key type asymmetric registered [ 3.110565] Asymmetric key parser 'x509' registered [ 3.112689] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.116230] io scheduler mq-deadline registered [ 3.119344] io scheduler kyber registered [ 3.121443] io scheduler bfq registered [ 3.123745] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.126932] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.129840] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.133116] ACPI: Power Button [PWRF] [ 3.140384] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.147975] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.162469] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.168532] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.181583] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.211270] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.238289] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.242284] Non-volatile memory driver v1.3 [ 3.244102] Linux agpgart interface v0.103 [ 3.278625] virtio_blk virtio1: [vda] 145912 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.281795] vda: detected capacity change from 0 to 74706944 [ 3.295400] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.298778] vdb: detected capacity change from 0 to 1073741824 [ 3.314841] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.318168] vdc: detected capacity change from 0 to 2621440000 [ 3.344228] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.349037] vdd: detected capacity change from 0 to 2621440000 [ 3.377637] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.380855] vde: detected capacity change from 0 to 4294967296 [ 3.402433] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.405892] vdf: detected capacity change from 0 to 4294967296 [ 3.422962] libphy: Fixed MDIO Bus: probed [ 3.429305] usbcore: registered new interface driver usbserial_generic [ 3.431650] usbserial: USB Serial support registered for generic [ 3.433929] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.439644] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.442055] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.445516] mousedev: PS/2 mouse device common for all mice [ 3.448493] rtc_cmos 00:05: RTC can wake from S4 [ 3.449446] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.453938] rtc_cmos 00:05: registered as rtc0 [ 3.457193] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.457730] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.460602] intel_pstate: CPU model not supported [ 3.465983] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.471071] hid: raw HID events driver (C) Jiri Kosina [ 3.473354] usbcore: registered new interface driver usbhid [ 3.475581] usbhid: USB HID core driver [ 3.477412] drop_monitor: Initializing network drop monitor service [ 3.480021] Initializing XFRM netlink socket [ 3.482273] NET: Registered protocol family 10 [ 3.486530] Segment Routing with IPv6 [ 3.487672] NET: Registered protocol family 17 [ 3.489875] mpls_gso: MPLS GSO support [ 3.496324] RAS: Correctable Errors collector initialized. [ 3.498629] AVX version of gcm_enc/dec engaged. [ 3.500547] AES CTR mode by8 optimization enabled [ 3.587840] sched_clock: Marking stable (3587762710, 0)->(4487913690, -900150980) [ 3.592218] registered taskstats version 1 [ 3.594227] Loading compiled-in X.509 certificates [ 3.595809] zswap: loaded using pool lzo/zbud [ 3.624755] Key type big_key registered [ 3.641542] Key type encrypted registered [ 3.643720] ima: No TPM chip found, activating TPM-bypass! [ 3.646286] ima: Allocated hash algorithm: sha1 [ 3.648468] ima: No architecture policies found [ 3.650603] evm: Initialising EVM extended attributes: [ 3.652776] evm: security.selinux [ 3.654407] evm: security.ima [ 3.655721] evm: security.capability [ 3.657307] evm: HMAC attrs: 0x1 [ 3.660252] rtc_cmos 00:05: setting system clock to 2026-08-14 18:27:53 UTC (1786732073) [ 3.668385] debug: unmapping init [mem 0xffffffffb9003000-0xffffffffb91fffff] [ 3.672120] debug: unmapping init [mem 0xffffffffb7d82000-0xffffffffb8058fff] [ 3.682109] Write protecting the kernel read-only data: 28672k [ 3.686467] debug: unmapping init [mem 0xffffffffb6403000-0xffffffffb65fffff] [ 3.689355] debug: unmapping init [mem 0xffffffffb6d14000-0xffffffffb6dfffff] [ 3.733155] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.739886] systemd[1]: Detected virtualization kvm. [ 3.741481] systemd[1]: Detected architecture x86-64. [ 3.742746] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.769951] systemd[1]: No hostname configured. [ 3.771047] systemd[1]: Set hostname to . [ 3.772462] random: systemd: uninitialized urandom read (16 bytes read) [ 3.773922] systemd[1]: Initializing machine ID from random generator. [ 3.899238] random: systemd: uninitialized urandom read (16 bytes read) [ 3.900732] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.903834] random: systemd: uninitialized urandom read (16 bytes read) [ 3.905729] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.908474] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.575916] device-mapper: uevent: version 1.0.3 [ 4.578285] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK [0[ 4.825448] random: fast init done m] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.374641] virtio_net virtio0 ens2: renamed from eth0 [ 5.481626] scsi host0: ata_piix [ 5.508553] scsi host1: ata_piix [ 5.510273] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.512714] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.946344] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.162448] random: crng init done [ 10.163964] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.686061] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.903448] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.144948] SELinux: Disabled at runtime. [ 12.212499] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.220716] systemd[1]: Detected virtualization kvm. [ 12.222713] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.727438] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.730022] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.734123] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.736924] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.739902] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.750518] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.770878] systemd[1]: Listening on Process Core Dump Socket. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on RPCbind Server Activation Socket. Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd File Systems. Mounting Huge Pages File System... Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ 12.871876] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Stopped target Initrd Root File System. Starting Create list of required st…ce nodes for the current kernel... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting udev Coldplug all Devices... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.259523] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.538586] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.685195] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.769202] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.792756] EDAC sbridge: Ver: 1.1.2 [ 15.289500] Key type dns_resolver registered [ 15.583334] NFS: Registering the id_resolver key type [ 15.585418] Key type id_resolver registered [ 15.587106] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg211-server login: [ 41.103224] libcfs: loading out-of-tree module taints kernel. [ 41.125786] Key type ._llcrypt registered [ 41.127316] Key type .llcrypt registered [ 41.185556] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 53.761454] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 55.478890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 55.495305] alg: No test for adler32 (adler32-zlib) [ 56.947180] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 57.884548] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 59.690064] Key type lgssc registered [ 61.528521] Lustre: Echo OBD driver; http://www.lustre.org/ [ 75.528009] hrtimer: interrupt took 3791334 ns [ 80.661338] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 128.023310] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 141.734312] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 141.750257] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 142.956465] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 142.987726] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 143.056411] Lustre: lustre-MDT0000: new disk, initializing [ 143.137181] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 143.167535] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 147.592076] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 161.785810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 161.855111] Lustre: 6486:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 161.879190] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 161.883206] Lustre: Skipped 1 previous similar message [ 161.951709] Lustre: lustre-MDT0001: new disk, initializing [ 162.032496] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 162.068722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 162.090688] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 166.641431] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 171.407968] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 182.051068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 182.301992] Lustre: lustre-OST0000: new disk, initializing [ 182.305385] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 182.308628] Lustre: 8421:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 182.353720] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 183.136077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 183.156524] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 183.257442] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 188.426895] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 201.957633] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 202.208432] Lustre: lustre-OST0001: new disk, initializing [ 202.215707] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 202.219350] Lustre: 9493:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 202.324026] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 208.941596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 208.964381] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 209.048171] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 210.156763] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 223.455384] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 229.557048] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 237.210577] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing check_logdir /tmp/testlogs/ [ 243.030024] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing yml_node [ 249.590839] Lustre: DEBUG MARKER: Client: 2.17.56.2 [ 253.261321] Lustre: DEBUG MARKER: MDS: 2.17.56.2 [ 256.827319] Lustre: DEBUG MARKER: OSS: 2.17.56.2 [ 259.626660] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Aug 14 14:32:07 EDT 2026 [ 279.192598] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 281.111662] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 300.517604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 300.530522] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 300.554199] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 303.378412] Lustre: server umount lustre-MDT0000 complete [ 306.805370] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732376 with bad export cookie 13774874370730093483 [ 306.806042] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 306.811406] LustreError: 6478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 311.265828] LustreError: 6493:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 311.272181] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 311.278535] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 311.278544] Lustre: Skipped 4 previous similar messages [ 311.280109] LustreError: 6493:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 311.333196] Lustre: Skipped 4 previous similar messages [ 313.131477] Lustre: server umount lustre-MDT0001 complete [ 317.084992] Lustre: server umount lustre-OST0000 complete [ 321.260845] Lustre: server umount lustre-OST0001 complete [ 328.172847] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 339.289753] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 373.635976] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 373.903802] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 373.946620] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 374.026187] Lustre: lustre-MDT0000: new disk, initializing [ 374.139813] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 374.176269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 378.433438] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 389.794484] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 390.021827] Lustre: 15050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 390.189610] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 390.195056] Lustre: Skipped 1 previous similar message [ 390.340255] Lustre: lustre-MDT0001: new disk, initializing [ 390.470245] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 390.542268] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 390.563962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 396.115903] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 408.320310] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 415.860756] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 423.275510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 423.711945] Lustre: lustre-OST0000: new disk, initializing [ 423.723453] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 423.732510] Lustre: 17021:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 423.841328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 425.729444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 425.751650] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 425.880367] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 431.713920] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 443.134400] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 447.480495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 447.689454] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 452.594646] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 452.768180] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 456.672962] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 456.673718] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 456.681681] LustreError: Skipped 1 previous similar message [ 456.690578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 456.699027] Lustre: Skipped 1 previous similar message [ 456.740911] Lustre: Skipped 2 previous similar messages [ 460.639636] Lustre: server umount lustre-OST0000 complete [ 466.913381] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 466.915315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 466.928806] Lustre: Skipped 1 previous similar message [ 466.940620] Lustre: Skipped 1 previous similar message [ 471.156669] Lustre: server umount lustre-MDT0000 complete [ 472.032609] LustreError: 15060:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 472.058939] LustreError: 15060:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 474.617205] LustreError: 15041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732544 with bad export cookie 13774874370730095016 [ 474.618635] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 474.624892] LustreError: 15041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 475.112824] Lustre: server umount lustre-MDT0001 complete [ 480.239295] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 14:35:48 (1786732548) [ 486.542067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 486.875615] LustreError: 19285:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 486.931229] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 491.027056] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 492.001971] LustreError: 19286:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 497.123392] LustreError: 19285:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 499.003636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 503.205653] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 510.154887] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 515.833122] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 522.329419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 528.123320] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 534.747111] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 538.134795] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 545.761760] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 545.768549] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.778895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 550.677353] Lustre: server umount lustre-MDT0000 complete [ 552.421289] LustreError: 19285:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 552.447871] LustreError: 19285:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 554.124337] LustreError: 19267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732623 with bad export cookie 13774874370730095891 [ 554.132603] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 554.145212] LustreError: 19267:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 554.459095] Lustre: server umount lustre-MDT0001 complete [ 567.610989] Lustre: server umount lustre-OST0000 complete [ 587.176735] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 598.767373] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 599.239256] LustreError: 23679:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 599.351661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 599.356793] Lustre: Skipped 2 previous similar messages [ 603.807593] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 613.896376] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 618.658636] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 621.406163] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 627.469240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 633.712131] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 642.391428] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 642.568695] Lustre: lustre-OST0001: new disk, initializing [ 642.574188] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 642.587907] Lustre: 25994:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 644.596066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 644.603618] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 644.779260] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 648.522838] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 659.074993] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 664.014278] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 676.845678] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 676.847691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 676.861023] Lustre: Skipped 5 previous similar messages [ 676.876740] Lustre: Skipped 6 previous similar messages [ 682.460453] Lustre: server umount lustre-MDT0000 complete [ 686.322795] LustreError: 23659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732756 with bad export cookie 13774874370730096738 [ 686.327213] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 686.329565] LustreError: 23659:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 686.695612] Lustre: server umount lustre-MDT0001 complete [ 701.055955] Lustre: server umount lustre-OST0000 complete [ 702.415588] Lustre: 3618:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786732756/real 1786732756] req@ffff9fda036f1180 x1873524427049088/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786732772 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 702.463898] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 704.817406] Lustre: server umount lustre-OST0001 complete [ 720.709602] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 731.797027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 732.130616] LustreError: 29015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 732.149362] LustreError: 29015:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 732.252789] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 732.262789] Lustre: Skipped 3 previous similar messages [ 737.130109] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 746.725603] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.344938] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 754.248904] Lustre: 30163:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 761.018405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 768.152655] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 777.020637] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 782.962090] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 791.037887] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 795.155288] Lustre: 32042:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 802.787169] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 802.792718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 802.797908] Lustre: Skipped 3 previous similar messages [ 802.804808] Lustre: Skipped 7 previous similar messages [ 809.245390] Lustre: server umount lustre-MDT0000 complete [ 813.032852] LustreError: 29015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 813.054406] LustreError: 29015:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 813.090974] LustreError: 28997:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786732882 with bad export cookie 13774874370730098040 [ 813.096547] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 813.105139] LustreError: 28997:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 813.581268] Lustre: server umount lustre-MDT0001 complete [ 828.734267] Lustre: server umount lustre-OST0000 complete [ 833.312092] Lustre: 3618:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786732887/real 1786732887] req@ffff9fdb36aee300 x1873524427100032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786732903 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 833.681642] Lustre: server umount lustre-OST0001 complete [ 843.296214] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 14:41:51 (1786732911) [ 871.761975] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 881.258558] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 885.487356] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 893.319314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 897.494109] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 900.950328] Lustre: 35992:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 907.574809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 914.789585] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 923.939695] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 931.034500] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 940.793475] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 944.163777] Lustre: 37871:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 950.241422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 950.246434] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 950.260459] Lustre: Skipped 1 previous similar message [ 950.268122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 950.274533] Lustre: Skipped 4 previous similar messages [ 955.333261] Lustre: server umount lustre-MDT0000 complete [ 956.904975] LustreError: 35578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 956.948756] LustreError: 35578:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 958.442118] LustreError: 35994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733028 with bad export cookie 13774874370730099146 [ 958.452711] LustreError: 35994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 958.454619] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 958.841831] Lustre: server umount lustre-MDT0001 complete [ 971.491151] Lustre: server umount lustre-OST0000 complete [ 985.057529] Lustre: server umount lustre-OST0001 complete [ 994.673914] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 14:44:22 (1786733062) [ 1012.504684] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1022.345703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1022.651899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1022.661231] Lustre: Skipped 7 previous similar messages [ 1026.188376] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1034.588850] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1039.662171] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1042.830028] Lustre: 41165:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1051.306698] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1058.737491] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1069.121694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1077.310761] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1085.213149] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1088.297569] Lustre: 43045:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1089.196486] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1089.199016] Lustre: Skipped 1 previous similar message [ 1095.172413] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1095.178148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1095.182103] Lustre: Skipped 4 previous similar messages [ 1095.193233] Lustre: Skipped 4 previous similar messages [ 1098.814394] Lustre: server umount lustre-MDT0000 complete [ 1102.055160] LustreError: 39998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733171 with bad export cookie 13774874370730100252 [ 1102.083405] LustreError: 39998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1102.087982] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1108.686293] Lustre: server umount lustre-MDT0001 complete [ 1119.140503] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1126.882801] LustreError: 3617:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9fdb3b2c7b80 x1873524427203456/t0(0) o250->MGC192.168.202.111@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1133.864446] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1136.968761] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1143.796498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1149.137695] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1149.430986] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1149.436810] Lustre: Skipped 1 previous similar message [ 1157.884762] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1159.483301] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1165.303469] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1166.898308] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1175.030620] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1178.488190] Lustre: server umount lustre-MDT0000 complete [ 1182.545835] LustreError: 39998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733252 with bad export cookie 13774874370730101344 [ 1182.552641] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1182.558069] LustreError: 39998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1182.869277] Lustre: server umount lustre-MDT0001 complete [ 1197.605609] Lustre: server umount lustre-OST0000 complete [ 1211.785826] Lustre: server umount lustre-OST0001 complete [ 1224.982148] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 14:48:13 (1786733293) [ 1245.059504] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1254.850889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1255.210881] LustreError: 47653:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1255.235661] LustreError: 47653:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 31 previous similar messages [ 1260.306738] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1269.185355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1273.768524] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1276.608774] Lustre: 48806:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1281.887787] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1288.796065] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1296.498727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1303.074641] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1311.841176] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1316.935676] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1328.329867] Lustre: server umount lustre-MDT0000 complete [ 1331.178332] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1331.873817] LustreError: 47634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786733401 with bad export cookie 13774874370730102170 [ 1331.882544] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1331.890310] LustreError: 47634:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1332.157403] Lustre: server umount lustre-MDT0001 complete [ 1347.266688] Lustre: server umount lustre-OST0000 complete [ 1361.655241] Lustre: server umount lustre-OST0001 complete [ 1372.713196] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 14:50:40 (1786733440) [ 1375.467877] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1377.903607] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 14:50:45 (1786733445) [ 1388.763529] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1398.177016] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1422.009928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1422.270606] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1422.333489] Lustre: lustre-MDT0000: new disk, initializing [ 1422.408728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1426.240382] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1435.978699] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1443.572528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1443.850396] Lustre: lustre-OST0000: new disk, initializing [ 1443.853479] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1443.857957] Lustre: Skipped 1 previous similar message [ 1443.861274] Lustre: 54883:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1445.668677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1445.681592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1445.711304] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1451.021620] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1463.993884] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1467.066987] Lustre: DEBUG MARKER: create test files [ 1507.300812] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1507.305152] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1507.312378] Lustre: Skipped 12 previous similar messages [ 1507.320026] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1507.325177] Lustre: Skipped 13 previous similar messages [ 1509.134031] Lustre: server umount lustre-OST0000 complete [ 1513.312286] Lustre: server umount lustre-MDT0000 complete [ 1526.188740] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1528.809607] Key type lgssc unregistered [ 1529.143762] LNet: 56512:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1529.149922] LNetError: 56512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1530.219850] LNet: Removed LNI 192.168.202.111@tcp [ 1531.186199] Key type .llcrypt unregistered [ 1531.188894] Key type ._llcrypt unregistered [ 1560.895254] Key type ._llcrypt registered [ 1560.902717] Key type .llcrypt registered [ 1561.027100] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1562.095919] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1562.298365] alg: No test for adler32 (adler32-zlib) [ 1563.385923] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 1563.667448] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1565.391898] Key type lgssc registered [ 1566.407802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1575.011625] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1575.037439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1576.678428] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1581.450084] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1589.472349] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1598.671805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1599.096961] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1603.188675] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1603.279167] Lustre: 59242:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1603.297543] Lustre: 59242:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1603.307553] Lustre: 59242:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1603.313493] Lustre: 59242:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1603.318701] Lustre: 59242:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1603.323483] Lustre: 59242:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1606.913468] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1617.426348] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1624.669740] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1631.934696] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1641.748910] Lustre: server umount lustre-OST0000 complete [ 1655.450588] Lustre: server umount lustre-MDT0000 complete [ 1666.896709] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1669.363737] Key type lgssc unregistered [ 1669.622647] LNet: 60852:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1669.634224] LNetError: 60852:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1669.651084] LNet: Removed LNI 192.168.202.111@tcp [ 1670.303962] Key type .llcrypt unregistered [ 1670.308182] Key type ._llcrypt unregistered [ 1701.428540] Key type ._llcrypt registered [ 1701.433492] Key type .llcrypt registered [ 1701.529272] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1702.513428] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1702.526196] alg: No test for adler32 (adler32-zlib) [ 1703.538967] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 1703.749449] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1705.424458] Key type lgssc registered [ 1706.887975] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1715.263847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1715.290874] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1716.687707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1720.707582] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1728.456152] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1734.934813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1735.210557] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1741.068858] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1741.358392] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1741.437246] Lustre: 63666:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1741.443501] Lustre: 63666:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1741.451204] Lustre: 63666:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1741.458332] Lustre: 63666:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1741.466618] Lustre: 63666:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1741.472968] Lustre: 63666:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1747.982190] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1753.718381] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1766.880216] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1766.890287] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1766.907396] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1769.644228] Lustre: server umount lustre-OST0000 complete [ 1775.329567] Lustre: server umount lustre-MDT0000 complete [ 1788.612723] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 1791.266392] Key type lgssc unregistered [ 1791.530685] LNet: 65226:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1791.535645] LNetError: 65226:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1791.546498] LNet: Removed LNI 192.168.202.111@tcp [ 1792.334875] Key type .llcrypt unregistered [ 1792.337364] Key type ._llcrypt unregistered [ 1811.288370] Key type ._llcrypt registered [ 1811.291208] Key type .llcrypt registered [ 1811.410625] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 1828.946264] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 1830.462353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1830.601527] alg: No test for adler32 (adler32-zlib) [ 1831.689404] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 1831.991076] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 1833.695420] Key type lgssc registered [ 1834.817747] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1869.987847] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1870.007304] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1871.283185] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1871.310782] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1871.402815] Lustre: lustre-MDT0000: new disk, initializing [ 1871.528294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1871.555517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1875.201805] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1885.057951] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1885.210773] Lustre: 68380:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1885.260800] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1885.267377] Lustre: Skipped 1 previous similar message [ 1885.350903] Lustre: lustre-MDT0001: new disk, initializing [ 1885.422926] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1885.450735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1885.457352] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1889.491839] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1899.640554] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1905.948792] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1912.788254] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1913.128659] Lustre: lustre-OST0000: new disk, initializing [ 1913.134173] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1913.142783] Lustre: 70348:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1913.250683] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1919.647332] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 1920.607129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1920.621881] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1920.794607] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1930.324187] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1935.455864] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1935.655914] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1940.468514] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1940.682140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1946.601095] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1946.602198] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1946.603182] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1946.617184] Lustre: Skipped 1 previous similar message [ 1946.629617] LustreError: Skipped 1 previous similar message [ 1951.713590] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1951.728157] Lustre: Skipped 2 previous similar messages [ 1956.831791] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1956.839144] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1956.846810] Lustre: Skipped 1 previous similar message [ 1957.120611] Lustre: server umount lustre-OST0000 complete [ 1961.952774] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1961.963491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1961.967671] Lustre: Skipped 1 previous similar message [ 1961.981671] Lustre: Skipped 1 previous similar message [ 1967.138966] Lustre: server umount lustre-MDT0000 complete [ 1972.207883] LustreError: 68389:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1972.231615] LustreError: 68389:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 1972.742310] LustreError: 68373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734042 with bad export cookie 12057332979372946599 [ 1972.752711] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1972.763642] LustreError: 68373:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1974.302688] Lustre: server umount lustre-MDT0001 complete [ 1988.764467] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 15:00:56 (1786734056) [ 2002.263523] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2041.314539] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2051.291829] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2087.386053] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2087.588033] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2087.607309] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2087.663150] Lustre: lustre-MDT0000: new disk, initializing [ 2087.720290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2087.733348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2091.766079] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2102.371927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2102.568889] Lustre: 76390:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2102.638271] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2102.641327] Lustre: Skipped 1 previous similar message [ 2102.704815] Lustre: lustre-MDT0001: new disk, initializing [ 2102.813197] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2102.853664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2102.867782] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2107.237445] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2116.065208] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2122.638697] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2131.212498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2131.742766] Lustre: lustre-OST0000: new disk, initializing [ 2131.745968] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2131.751768] Lustre: 78358:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2131.838823] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2138.326872] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2138.777087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2138.801408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2138.911522] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2148.174270] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2152.479125] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2152.696068] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2157.318917] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2157.553394] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2164.704073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2164.705407] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2164.710863] LustreError: Skipped 1 previous similar message [ 2164.713591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2164.725510] Lustre: Skipped 1 previous similar message [ 2166.172603] Lustre: server umount lustre-OST0000 complete [ 2174.434313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2174.442411] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2174.464094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2174.469915] Lustre: Skipped 1 previous similar message [ 2177.092252] Lustre: server umount lustre-MDT0000 complete [ 2180.071027] LustreError: 76401:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2180.095819] LustreError: 76401:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2181.365136] LustreError: 76380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734251 with bad export cookie 12057332979372947481 [ 2181.381715] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2181.385148] LustreError: 76380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2181.999571] Lustre: server umount lustre-MDT0001 complete [ 2193.347582] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 15:04:21 (1786734261) [ 2199.635376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2200.284710] LustreError: 80772:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2200.442756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2204.906990] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2205.674608] LustreError: 80773:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2210.794820] LustreError: 80772:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2213.789424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2218.326215] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2226.535933] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2232.588547] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2240.697331] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2241.068819] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2241.077780] Lustre: Skipped 1 previous similar message [ 2247.749654] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2254.175282] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2257.584169] Lustre: 83442:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 638f9c5e-e65a-48fa-8a9a-5290dd4610bf at adminstrative request [ 2258.478088] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2258.504155] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2268.646925] Lustre: MGS: Client 638f9c5e-e65a-48fa-8a9a-5290dd4610bf (at 0@lo) reconnecting [ 2268.658485] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xa754407656207b8b to 0xa754407656207e2b [ 2268.681589] Lustre: MGC192.168.202.111@tcp: Connection restored to 0@lo (at 0@lo) [ 2294.427273] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2294.588324] Lustre: lustre-OST0001: new disk, initializing [ 2294.599178] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2294.609831] Lustre: 83833:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2300.607879] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2304.653098] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2304.669445] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2304.831907] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2310.704319] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2315.234033] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2315.245388] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2315.260900] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2315.272867] Lustre: Skipped 2 previous similar messages [ 2315.278357] Lustre: Skipped 1 previous similar message [ 2329.055258] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2329.178415] Lustre: server umount lustre-OST0001 complete [ 2330.611918] LustreError: 82592:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2330.658773] LustreError: 82592:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2335.202447] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2335.212968] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2340.100530] Lustre: server umount lustre-OST0000 complete [ 2351.074849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2351.079550] Lustre: Skipped 10 previous similar messages [ 2351.802263] Lustre: server umount lustre-MDT0000 complete [ 2355.173807] LustreError: 82000:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2355.206403] LustreError: 82000:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2356.163939] LustreError: 80753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734425 with bad export cookie 12057332979372949035 [ 2356.163939] LustreError: 80754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734425 with bad export cookie 12057332979372949035 [ 2356.163950] LustreError: 80753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2356.165888] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2356.178355] LustreError: 80754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2356.833392] Lustre: server umount lustre-MDT0001 complete [ 2366.070349] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 15:07:14 (1786734434) [ 2367.585760] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2369.295765] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 15:07:17 (1786734437) [ 2370.711788] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2372.462811] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 15:07:20 (1786734440) [ 2374.346063] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2376.492406] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 15:07:24 (1786734444) [ 2378.327189] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2380.169693] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 15:07:28 (1786734448) [ 2381.765068] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2388.226818] loop: module loaded [ 2388.259874] loop0: detected capacity change from 0 to 4294967296 [ 2388.354760] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2390.072974] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2393.415478] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2398.485504] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2422.797340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2422.840774] LustreError: 87963:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2422.851688] LustreError: 87963:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2422.862891] LustreError: 87963:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2422.874669] LustreError: 87963:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2441.182826] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 2451.235196] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 2486.747081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2487.051156] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2487.084269] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2487.156977] Lustre: lustre-MDT0000: new disk, initializing [ 2487.226313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2487.231599] Lustre: Skipped 1 previous similar message [ 2487.246298] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2491.750318] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2501.931157] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2502.067943] Lustre: 90829:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2502.314751] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2507.283303] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2517.518224] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2524.235368] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2531.632450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2531.917928] Lustre: 92796:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2539.091977] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2539.209264] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2540.465677] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2551.131954] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2557.200495] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2557.421203] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2561.934777] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2562.151241] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2565.098456] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2565.100915] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2565.118490] LustreError: Skipped 1 previous similar message [ 2565.119842] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2565.146585] Lustre: Skipped 4 previous similar messages [ 2565.172308] Lustre: Skipped 1 previous similar message [ 2577.887157] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2578.032754] Lustre: server umount lustre-OST0000 complete [ 2584.544226] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2587.638611] Lustre: server umount lustre-MDT0000 complete [ 2590.694726] LustreError: 91822:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2590.721645] LustreError: 91822:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2591.682222] LustreError: 91622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734661 with bad export cookie 12057332979372949651 [ 2591.684624] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2591.695182] LustreError: 91622:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2592.141218] Lustre: server umount lustre-MDT0001 complete [ 2600.328958] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 15:11:08 (1786734668) [ 2605.952889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2610.132290] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2617.653666] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2618.120206] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2618.125665] Lustre: Skipped 3 previous similar messages [ 2622.673282] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2630.754932] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2637.991277] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2645.042836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2652.880424] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2660.606793] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2666.466414] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2666.724676] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2672.381793] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2672.642265] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2680.903436] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2681.196434] Lustre: lustre-OST0001: new disk, initializing [ 2681.205110] Lustre: Skipped 2 previous similar messages [ 2681.211884] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2681.218639] Lustre: Skipped 3 previous similar messages [ 2681.229974] Lustre: 98580:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2687.636233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2687.642699] Lustre: Skipped 2 previous similar messages [ 2687.655696] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2687.831400] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2689.199576] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2700.111812] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2704.815736] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2705.036065] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2709.129048] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2709.510732] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2729.068642] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2755.609862] Lustre: Failing over lustre-MDT0000 [ 2756.069245] Lustre: server umount lustre-MDT0000 complete [ 2756.577532] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2756.586760] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2756.603022] Lustre: Skipped 2 previous similar messages [ 2756.611071] LustreError: 95211:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2756.633576] LustreError: 95211:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2776.095247] Lustre: 66605:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786734829/real 1786734829] req@000000004c07e274 x1873526287256704/t0(0) o400->MGC192.168.202.111@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786734845 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2776.120547] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2781.656536] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2781.659284] LDISKFS-fs (dm-0): recovery complete [ 2781.666594] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2785.666514] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2788.372788] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2788.379675] Lustre: 95207:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2790.618858] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2790.888723] Lustre: 95210:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2790.896567] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2790.903346] Lustre: 95210:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 2791.038145] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2791.372047] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2791.700442] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2791.740761] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2792.072451] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2792.077156] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2792.095033] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2792.106596] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 2792.752567] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2792.766030] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2793.103058] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2793.125975] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2794.120160] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2794.130065] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 2794.810538] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2794.827036] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2795.143423] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2795.152630] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 2798.360488] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2798.372329] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2799.015055] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2799.030547] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2799.359177] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2799.367225] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2806.458660] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2806.469780] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 2807.112893] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2807.136493] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2807.479174] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2807.483507] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2822.741798] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2822.758262] Lustre: 101428:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 50 previous similar messages [ 2823.400208] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2823.409106] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2823.743199] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2823.762846] LustreError: 101428:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 2848.079216] Lustre: 101428:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2848.095737] Lustre: 101428:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2848.112683] Lustre: 101428:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 8a3563f0-52dd-451c-ae70-42b105152667@192.168.202.11@tcp [ 2848.119385] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2848.123792] Lustre: 101428:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000cdf31a3f x1873526269219968/t0(8589934774) o35->8a3563f0-52dd-451c-ae70-42b105152667@192.168.202.11@tcp:248/0 lens 392/0 e 0 to 0 dl 1786734928 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2848.164651] Lustre: lustre-MDT0000: Denying connection for new client 8a3563f0-52dd-451c-ae70-42b105152667 (at 192.168.202.11@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2848.168542] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2848.205442] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2848.257366] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2848.269553] Lustre: Skipped 2 previous similar messages [ 2848.273667] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2848.341562] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 2848.343118] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2867.169960] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2867.183722] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2867.193283] Lustre: Skipped 3 previous similar messages [ 2867.197495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2867.203353] Lustre: Skipped 7 previous similar messages [ 2871.245095] Lustre: server umount lustre-OST0000 complete [ 2888.164676] LustreError: 97032:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2888.189839] LustreError: 97032:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 49 previous similar messages [ 2890.207614] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2890.609497] Lustre: server umount lustre-OST0001 complete [ 2909.151367] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2909.365419] Lustre: server umount lustre-MDT0000 complete [ 2913.436890] LustreError: 95191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786734983 with bad export cookie 12057332979373000870 [ 2913.450571] LustreError: 95191:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2913.451649] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2913.903417] Lustre: server umount lustre-MDT0001 complete [ 2924.005417] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 15:16:32 (1786734992) [ 2933.058856] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2933.074599] Lustre: *** cfs_fail_loc=197, val=0*** [ 2948.640356] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2953.760215] LustreError: 103409:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 2958.910541] Lustre: *** cfs_fail_loc=197, val=0*** [ 2958.912887] Lustre: Skipped 65 previous similar messages [ 2979.487268] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2979.494142] Lustre: Skipped 3 previous similar messages [ 2985.265875] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 2993.111922] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2995.487808] Lustre: server umount lustre-OST0000 complete [ 3004.833392] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 15:17:53 (1786735073) [ 3026.640692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3032.044426] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3041.473872] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3046.222403] LustreError: 105061:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3046.251279] LustreError: 105061:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3046.264896] LustreError: 105061:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3046.956032] Lustre: server umount lustre-MDT0000 complete [ 3083.971181] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3102.365964] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3145.389309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3145.616322] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3145.654969] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3145.791562] Lustre: lustre-MDT0000: new disk, initializing [ 3145.906579] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3151.761828] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3168.224505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3168.376747] Lustre: 109256:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3168.811112] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3175.172616] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3186.557837] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3193.033735] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3200.918758] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3201.223539] Lustre: 111225:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3208.209608] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3209.844563] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3217.484494] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3222.755546] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3222.944978] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3228.216418] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3228.462074] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3230.690883] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3230.713730] LustreError: Skipped 1 previous similar message [ 3230.720520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3230.729042] Lustre: Skipped 5 previous similar messages [ 3230.736845] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3230.745699] Lustre: Skipped 13 previous similar messages [ 3236.734114] Lustre: server umount lustre-OST0000 complete [ 3255.263497] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3255.503110] Lustre: server umount lustre-MDT0000 complete [ 3255.789968] LustreError: 109267:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3255.823354] LustreError: 109267:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3260.119588] LustreError: 109248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786735329 with bad export cookie 12057332979373011503 [ 3260.124579] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3260.137624] LustreError: 109248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3260.731383] Lustre: server umount lustre-MDT0001 complete [ 3272.802786] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 15:22:20 (1786735340) [ 3285.911979] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3289.530356] Key type lgssc unregistered [ 3289.955384] LNet: 113716:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3289.974603] LNetError: 113716:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3290.002197] LNet: Removed LNI 192.168.202.111@tcp [ 3291.019942] Key type .llcrypt unregistered [ 3291.021693] Key type ._llcrypt unregistered [ 3314.892408] Key type ._llcrypt registered [ 3314.896102] Key type .llcrypt registered [ 3315.034422] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3334.957868] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3336.248447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3336.328970] alg: No test for adler32 (adler32-zlib) [ 3337.550695] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 3337.784229] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3339.455633] Key type lgssc registered [ 3341.217460] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3385.285584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3385.303175] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3386.627478] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3386.681661] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3386.801218] Lustre: lustre-MDT0000: new disk, initializing [ 3386.865489] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3386.879453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3391.528927] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3401.766845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3402.149727] Lustre: lustre-OST0000: new disk, initializing [ 3402.160536] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3402.171992] Lustre: Skipped 1 previous similar message [ 3402.220691] Lustre: 117892:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3402.335728] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3403.368991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3403.380489] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3403.422272] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3409.454115] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3421.214422] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3421.415594] Lustre: lustre-OST0001: new disk, initializing [ 3421.420383] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3421.437378] Lustre: 118755:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3421.541258] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3427.103191] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3428.955814] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3428.966249] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3428.993954] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3439.079036] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3439.110583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3444.005176] Lustre: server umount lustre-MDT0000 complete [ 3450.325816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3456.323320] LustreError: 117080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786735526 with bad export cookie 4444373151503460913 [ 3456.325959] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3456.331685] LustreError: 117080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3456.410736] Lustre: server umount lustre-OST0000 complete [ 3460.378415] Lustre: server umount lustre-OST0001 complete [ 3470.540508] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 15:25:38 (1786735538) [ 3496.560810] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 15:26:04 (1786735564) [ 3508.323724] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3536.736391] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 15:26:44 (1786735604) [ 3587.998251] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 3602.079631] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3641.340056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3641.559347] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3641.578972] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3641.649875] Lustre: lustre-MDT0000: new disk, initializing [ 3641.736437] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3641.764242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3646.533138] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3657.367519] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3657.443357] Lustre: 126512:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3657.535810] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3657.541601] Lustre: Skipped 1 previous similar message [ 3657.631795] Lustre: lustre-MDT0001: new disk, initializing [ 3657.736972] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3657.775490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3657.791137] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3661.565718] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3671.724449] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3679.203130] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3687.601732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3687.952428] Lustre: lustre-OST0000: new disk, initializing [ 3687.958316] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3687.964594] Lustre: 128481:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3688.076181] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3693.893767] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3695.737711] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3695.754885] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3695.881267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3706.666685] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3713.579724] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3714.069297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3720.801251] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3721.088065] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3726.828989] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3726.831079] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3726.840984] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3726.846938] Lustre: Skipped 2 previous similar messages [ 3726.855354] Lustre: Skipped 2 previous similar messages [ 3729.758463] Lustre: server umount lustre-OST0000 complete [ 3734.504039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3734.517372] LustreError: Skipped 1 previous similar message [ 3734.533947] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3734.559612] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3737.056315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3737.066040] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3742.180349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3742.190105] Lustre: Skipped 2 previous similar messages [ 3748.319131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3748.544401] Lustre: server umount lustre-MDT0000 complete [ 3752.430975] LustreError: 126504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786735822 with bad export cookie 4444373151503462138 [ 3752.432646] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3752.440033] LustreError: 126504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3752.881551] Lustre: server umount lustre-MDT0001 complete [ 3762.525751] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 15:30:30 (1786735830) [ 3769.209472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3769.663278] LustreError: 130896:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3769.700293] LustreError: 130896:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3769.741327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3774.322954] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3774.953222] LustreError: 130897:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3780.064936] LustreError: 130897:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3782.268725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3786.609333] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3794.321952] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3800.083875] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3806.362496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3806.782575] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3806.789401] Lustre: Skipped 1 previous similar message [ 3812.599794] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3820.518255] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3825.711571] Lustre: *** cfs_fail_loc=159, val=0*** [ 3842.179044] Lustre: lustre-MDT0000: Client 23c4d0cc-f8ea-48c4-ae72-46967791e89d (at 192.168.202.11@tcp) reconnecting [ 3843.890628] Lustre: *** cfs_fail_loc=159, val=0*** [ 3843.914697] Lustre: Skipped 5 previous similar messages [ 3859.528401] Lustre: lustre-MDT0000: Client 23c4d0cc-f8ea-48c4-ae72-46967791e89d (at 192.168.202.11@tcp) reconnecting [ 3879.392542] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3879.401847] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3879.420090] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3879.427226] Lustre: Skipped 2 previous similar messages [ 3881.441729] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3883.963693] Lustre: server umount lustre-OST0000 complete [ 3890.144568] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3890.158671] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3890.175807] Lustre: Skipped 1 previous similar message [ 3894.116646] Lustre: server umount lustre-MDT0000 complete [ 3896.808150] LustreError: 130897:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3896.862142] LustreError: 130897:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3898.707738] LustreError: 133619:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786735968 with bad export cookie 4444373151503463013 [ 3898.724441] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3898.973854] Lustre: server umount lustre-MDT0001 complete [ 3912.987056] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 3916.845187] Key type lgssc unregistered [ 3917.137792] LNet: 134674:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3917.143490] LNetError: 134674:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3917.157418] LNet: Removed LNI 192.168.202.111@tcp [ 3918.322845] Key type .llcrypt unregistered [ 3918.325120] Key type ._llcrypt unregistered [ 3949.495788] Key type ._llcrypt registered [ 3949.497906] Key type .llcrypt registered [ 3949.590314] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 3951.216791] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3951.589996] alg: No test for adler32 (adler32-zlib) [ 3953.091622] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 3953.430642] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 3955.136359] Key type lgssc registered [ 3957.505954] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3967.424410] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3967.437843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3968.754973] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3974.879698] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3978.215220] LustreError: 136092:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3978.221927] LustreError: 136092:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3983.349096] LustreError: 136091:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3985.369149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3985.782477] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3989.766469] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 3997.329401] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4002.562047] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4010.003699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4010.426496] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4013.535643] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4017.390422] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4025.300967] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4036.019694] Lustre: *** cfs_fail_loc=159, val=0*** [ 4051.519708] Lustre: lustre-MDT0000: Client bb1ab323-2d4c-4da4-a5c1-90f19414efff (at 192.168.202.11@tcp) reconnecting [ 4054.002397] Lustre: *** cfs_fail_loc=159, val=0*** [ 4054.014410] Lustre: Skipped 1 previous similar message [ 4069.486106] Lustre: lustre-MDT0001: Client bb1ab323-2d4c-4da4-a5c1-90f19414efff (at 192.168.202.11@tcp) reconnecting [ 4071.185709] Lustre: *** cfs_fail_loc=159, val=0*** [ 4071.192279] Lustre: Skipped 2 previous similar messages [ 4086.396684] Lustre: lustre-MDT0001: Client bb1ab323-2d4c-4da4-a5c1-90f19414efff (at 192.168.202.11@tcp) reconnecting [ 4103.402666] Lustre: 139154:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4105.976055] Lustre: *** cfs_fail_loc=159, val=0*** [ 4121.677231] Lustre: lustre-MDT0000: Client a1157be0-90e3-4417-ab53-ce086227afd9 (at 192.168.202.11@tcp) reconnecting [ 4123.502288] Lustre: *** cfs_fail_loc=159, val=0*** [ 4123.507311] Lustre: Skipped 14 previous similar messages [ 4140.140431] Lustre: lustre-MDT0000: Client a1157be0-90e3-4417-ab53-ce086227afd9 (at 192.168.202.11@tcp) reconnecting [ 4142.195439] Lustre: 139448:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4148.221989] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4148.223929] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4148.234518] LustreError: Skipped 1 previous similar message [ 4148.281483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4151.777302] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4151.780354] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4151.797301] LustreError: Skipped 4 previous similar messages [ 4151.797331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4151.838505] Lustre: Skipped 1 previous similar message [ 4154.120396] Lustre: server umount lustre-OST0000 complete [ 4159.969741] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4159.978108] LustreError: Skipped 6 previous similar messages [ 4159.982972] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4159.996724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4161.077644] Lustre: server umount lustre-MDT0000 complete [ 4162.016802] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4162.020400] LustreError: 139214:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4162.057294] LustreError: 139214:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4165.593985] LustreError: 136073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786736235 with bad export cookie 2797042044400215446 [ 4165.597355] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4165.606808] LustreError: 136073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4166.044081] Lustre: server umount lustre-MDT0001 complete [ 4183.279898] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4186.386131] Key type lgssc unregistered [ 4186.655245] LNet: 140530:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4186.661062] LNetError: 140530:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4186.691340] LNet: Removed LNI 192.168.202.111@tcp [ 4187.478602] Key type .llcrypt unregistered [ 4187.479858] Key type ._llcrypt unregistered [ 4220.555342] Key type ._llcrypt registered [ 4220.568233] Key type .llcrypt registered [ 4220.729413] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4222.289755] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4222.329063] alg: No test for adler32 (adler32-zlib) [ 4223.329739] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 4223.519234] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4225.199174] Key type lgssc registered [ 4226.131746] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4234.218775] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4234.242513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4235.961746] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4241.161143] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4245.480806] LustreError: 141945:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4245.510097] LustreError: 141945:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4250.595388] LustreError: 141946:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4251.056081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4251.639961] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4256.340478] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4264.333762] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4271.143339] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4278.549984] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4278.808834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4280.899106] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4284.212621] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4291.358230] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4296.263397] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4300.487919] Lustre: 144528:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4305.138966] Lustre: 144576:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4307.018550] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4308.024785] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4312.544575] Lustre: 144723:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4313.303726] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4314.301854] Lustre: 144822:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4321.763924] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4321.764487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4321.769286] LustreError: Skipped 1 previous similar message [ 4321.781537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4321.799288] Lustre: Skipped 1 previous similar message [ 4325.035555] Lustre: server umount lustre-OST0000 complete [ 4332.005537] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4332.023764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4332.030488] Lustre: Skipped 2 previous similar messages [ 4336.124915] Lustre: server umount lustre-MDT0000 complete [ 4337.122521] LustreError: 144686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4337.138751] LustreError: 144686:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4340.248241] LustreError: 143787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786736410 with bad export cookie 16543514891006546313 [ 4340.255615] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4340.258054] LustreError: 143787:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4340.790915] Lustre: server umount lustre-MDT0001 complete [ 4353.393517] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4356.629826] Key type lgssc unregistered [ 4356.946798] LNet: 145884:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4356.954615] LNetError: 145884:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4356.967276] LNet: Removed LNI 192.168.202.111@tcp [ 4358.221467] Key type .llcrypt unregistered [ 4358.225480] Key type ._llcrypt unregistered [ 4388.800851] Key type ._llcrypt registered [ 4388.802861] Key type .llcrypt registered [ 4388.928706] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4390.222143] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4390.384744] alg: No test for adler32 (adler32-zlib) [ 4391.441723] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 4391.733527] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4393.471192] Key type lgssc registered [ 4394.789159] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4402.915512] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4402.935680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4404.371212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4409.333312] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4414.945571] LustreError: 147300:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4414.966783] LustreError: 147300:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4419.010941] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4419.518652] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4424.352404] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4434.349627] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4441.638969] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4450.029686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4450.660847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4457.754101] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4465.027546] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4465.779803] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4474.230858] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4483.089547] Lustre: *** cfs_fail_loc=159, val=0*** [ 4498.492237] Lustre: lustre-MDT0000: Client d598483a-06b0-4c44-af53-cfad55ebc4eb (at 192.168.202.11@tcp) reconnecting [ 4505.065361] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4505.097363] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4505.098575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4510.178033] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4510.187165] Lustre: Skipped 2 previous similar messages [ 4511.111430] Lustre: server umount lustre-OST0000 complete [ 4515.307601] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4515.320837] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4515.326460] Lustre: Skipped 2 previous similar messages [ 4520.426717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4520.442473] Lustre: Skipped 2 previous similar messages [ 4520.709307] Lustre: server umount lustre-MDT0000 complete [ 4524.690189] LustreError: 147282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786736594 with bad export cookie 2483082666960798727 [ 4524.702886] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4524.706352] LustreError: 147282:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4525.432792] Lustre: server umount lustre-MDT0001 complete [ 4536.882871] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4540.447803] Key type lgssc unregistered [ 4540.787532] LNet: 150987:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4540.796151] LNetError: 150987:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4540.820740] LNet: Removed LNI 192.168.202.111@tcp [ 4542.035657] Key type .llcrypt unregistered [ 4542.037564] Key type ._llcrypt unregistered [ 4568.963245] Key type ._llcrypt registered [ 4568.965665] Key type .llcrypt registered [ 4569.098306] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4570.637454] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4570.807776] alg: No test for adler32 (adler32-zlib) [ 4571.859552] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 4572.129073] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4573.808164] Key type lgssc registered [ 4574.936343] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4582.657760] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4582.685176] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4584.443887] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4589.634863] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4594.659539] LustreError: 152402:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4594.688904] LustreError: 152402:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4598.631210] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4599.113483] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4603.623464] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4614.682650] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4622.394278] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4630.224050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4630.551378] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4636.267233] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4643.972243] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4644.811000] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4654.202968] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4674.428538] Lustre: lustre-OST0000: evicting fbb3f297-17bc-493b-8a2f-03ebf99fefd0 (at 192.168.202.11@tcp) by administrative request [ 4674.436902] Lustre: Skipped 1 previous similar message [ 4705.775612] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4705.790052] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4705.795051] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4705.810624] Lustre: Skipped 1 previous similar message [ 4705.825297] Lustre: Skipped 1 previous similar message [ 4709.989404] Lustre: server umount lustre-OST0000 complete [ 4716.005744] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4716.012526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4716.035251] Lustre: Skipped 1 previous similar message [ 4716.050439] Lustre: Skipped 1 previous similar message [ 4719.784943] Lustre: server umount lustre-MDT0000 complete [ 4721.120564] LustreError: 152402:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4721.136749] LustreError: 152402:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4723.768153] LustreError: 152384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786736793 with bad export cookie 1504459287206350836 [ 4723.770392] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4723.777952] LustreError: 152384:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4724.142895] Lustre: server umount lustre-MDT0001 complete [ 4735.468520] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4738.728897] Key type lgssc unregistered [ 4739.111049] LNet: 156494:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4739.127624] LNetError: 156494:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4739.168134] LNet: Removed LNI 192.168.202.111@tcp [ 4740.301585] Key type .llcrypt unregistered [ 4740.303773] Key type ._llcrypt unregistered [ 4820.446518] Key type ._llcrypt registered [ 4820.452782] Key type .llcrypt registered [ 4820.601447] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4821.602921] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4821.616423] alg: No test for adler32 (adler32-zlib) [ 4822.794723] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 4823.036845] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4824.719251] Key type lgssc registered [ 4825.716699] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4832.678532] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4832.709120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4834.138936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4839.283709] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4843.496594] LustreError: 158986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4843.511884] LustreError: 158986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4848.612668] LustreError: 158987:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4850.835879] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4851.715702] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4856.464363] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4866.227664] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4871.924420] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4878.098233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4878.315352] LustreError: 160813:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4878.528603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4883.964387] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4885.794520] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 4892.922829] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4896.255492] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4906.466219] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4906.470441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4906.484477] Lustre: Skipped 1 previous similar message [ 4906.493705] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4911.587955] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4911.611170] Lustre: Skipped 1 previous similar message [ 4912.838575] Lustre: server umount lustre-OST0000 complete [ 4921.825375] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4921.826740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4921.851989] Lustre: Skipped 1 previous similar message [ 4925.227478] Lustre: server umount lustre-MDT0000 complete [ 4926.946398] LustreError: 158987:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4926.978361] LustreError: 158987:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4930.682290] LustreError: 158968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786737000 with bad export cookie 15723204912545280798 [ 4930.703206] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4930.719951] LustreError: 158968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4931.047120] Lustre: server umount lustre-MDT0001 complete [ 4948.946394] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 4952.418284] Key type lgssc unregistered [ 4952.783740] LNet: 162572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4952.790142] LNetError: 162572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4953.841615] LNet: Removed LNI 192.168.202.111@tcp [ 4954.829478] Key type .llcrypt unregistered [ 4954.831298] Key type ._llcrypt unregistered [ 4983.030048] Key type ._llcrypt registered [ 4983.032425] Key type .llcrypt registered [ 4983.112555] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 4984.072376] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4984.162954] alg: No test for adler32 (adler32-zlib) [ 4985.225492] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 4985.428824] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 4987.103430] Key type lgssc registered [ 4988.933273] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5005.208239] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 15:51:13 (1786737073) [ 5021.622431] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 5032.687458] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5069.156836] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5069.195751] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5070.518105] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5070.577790] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5070.732934] Lustre: lustre-MDT0000: new disk, initializing [ 5070.860153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5070.887307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5075.244130] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5085.134721] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5085.217510] Lustre: 166833:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5085.265281] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5085.268331] Lustre: Skipped 1 previous similar message [ 5085.333682] Lustre: lustre-MDT0001: new disk, initializing [ 5085.407404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5085.459346] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5085.470702] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5089.262649] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5098.765216] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5104.074195] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5110.927957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5111.164393] Lustre: lustre-OST0000: new disk, initializing [ 5111.167718] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5111.173618] Lustre: 168804:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5111.284876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5117.853478] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5119.028351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5119.046397] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5119.182983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5126.499783] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5139.949145] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5139.956243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5139.957886] LustreError: Skipped 1 previous similar message [ 5139.959570] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5139.987712] Lustre: Skipped 1 previous similar message [ 5142.081447] Lustre: server umount lustre-OST0000 complete [ 5147.105220] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5147.112644] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5147.128772] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5147.135751] Lustre: Skipped 1 previous similar message [ 5150.183336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5150.184348] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5151.703181] Lustre: server umount lustre-MDT0000 complete [ 5156.320681] LustreError: 166827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786737226 with bad export cookie 10514337129432335935 [ 5156.340782] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5156.347146] LustreError: 166827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5157.016123] Lustre: server umount lustre-MDT0001 complete [ 5176.976278] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 5180.908335] Key type lgssc unregistered [ 5181.177052] LNet: 170732:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5181.182958] LNetError: 170732:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5181.200858] LNet: Removed LNI 192.168.202.111@tcp [ 5182.190296] Key type .llcrypt unregistered [ 5182.193830] Key type ._llcrypt unregistered [ 5208.834506] Key type ._llcrypt registered [ 5208.838458] Key type .llcrypt registered [ 5208.942485] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 5210.238854] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5210.317737] alg: No test for adler32 (adler32-zlib) [ 5211.367092] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 5211.534909] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 5213.175179] Key type lgssc registered [ 5214.117946] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5221.438205] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5221.458661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5222.933868] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5227.608316] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5233.142019] LustreError: 172148:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5233.164264] LustreError: 172148:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5236.866262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5237.347308] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5242.169689] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5252.442988] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5258.590681] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5266.364425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5266.795051] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5267.861139] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5275.784782] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 5286.191545] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5293.507637] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5293.553103] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5293.574737] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5293.609294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.636186] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5295.251511] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5295.300754] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5295.330326] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5295.359413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.373799] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5296.838078] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5296.852179] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5296.860856] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5296.866198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.873156] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5297.981752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.877231] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5298.881371] Lustre: Skipped 1 previous similar message [ 5298.899785] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5298.908906] Lustre: Skipped 1 previous similar message [ 5298.913349] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5298.919968] Lustre: Skipped 1 previous similar message [ 5298.924265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.933708] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5298.941640] Lustre: Skipped 1 previous similar message [ 5299.899071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.548966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.587516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.661050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.650608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5303.658976] Lustre: Skipped 4 previous similar messages [ 5303.684661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5303.705921] Lustre: Skipped 4 previous similar messages [ 5303.715490] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5303.723692] Lustre: Skipped 4 previous similar messages [ 5303.743557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.751586] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5303.754562] Lustre: Skipped 4 previous similar messages [ 5304.612965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.573588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.738290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.984079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.202819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.313400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.237949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.035532] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5312.043543] Lustre: Skipped 7 previous similar messages [ 5312.068263] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5312.084504] Lustre: Skipped 7 previous similar messages [ 5312.096031] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5312.105919] Lustre: Skipped 7 previous similar messages [ 5312.112477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.127328] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5312.130453] Lustre: Skipped 7 previous similar messages [ 5313.182192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.094567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.150763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.146179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.505329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.613669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.423415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.248791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.026103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.972059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.826736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.978179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.982473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.144599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.143268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.360417] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5328.366907] Lustre: Skipped 15 previous similar messages [ 5328.393744] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5328.401863] Lustre: Skipped 15 previous similar messages [ 5328.405510] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5328.407426] Lustre: Skipped 15 previous similar messages [ 5328.419539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.426261] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5328.434412] Lustre: Skipped 15 previous similar messages [ 5329.357464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.403121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.673698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.898970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.001272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.112730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.646902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.998864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.686908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.764866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.624289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.203825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.669393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.832989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.204787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.130795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.406484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.368787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.726043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.823352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.885491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.248071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.254798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.244938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.317448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.160564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.289807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.445731] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5360.464700] Lustre: Skipped 27 previous similar messages [ 5360.514440] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5360.544075] Lustre: Skipped 27 previous similar messages [ 5360.553208] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5360.564399] Lustre: Skipped 27 previous similar messages [ 5360.578874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.610728] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5360.632208] Lustre: Skipped 27 previous similar messages [ 5361.806041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.764218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.962776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.083769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.183081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.406900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.476524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.344112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.473948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.498867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.536163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.385457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.403225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.283147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.204606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.956453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.803881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.532185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.362483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.406966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.318351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.088372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.995484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.706831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.551490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.353399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.161898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.110208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.126387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.810620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.680711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.632560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.514332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.344512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.357275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.132397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.111397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.909960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.802776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.636612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.603308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.469355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.433565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.317298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.253792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.150258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.203860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.166427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.152948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.086602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.907146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.761272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.814933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.754257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.756537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.047768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.191675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.189329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.132751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.168379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.187654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.660233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.787678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.145069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.994480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.912882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.693966] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5424.699372] Lustre: Skipped 66 previous similar messages [ 5424.712447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5424.719560] Lustre: Skipped 66 previous similar messages [ 5424.723428] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5424.729211] Lustre: Skipped 66 previous similar messages [ 5424.735828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.747081] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5424.753728] Lustre: Skipped 66 previous similar messages [ 5425.733308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.672492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.627423] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5427.632633] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5427.633636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.452374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.425527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.327867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.063171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.760722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.854954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.164902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.228588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.252028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.230398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.154378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.038426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.863247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.833314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.754642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.763817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.775585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.681887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.487201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.346399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.152702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.177206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.055789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.870866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.650878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.560181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.424833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.372707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.050981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.816854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.350946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.975653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.783789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.614955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.608693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.656480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.295787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.365139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.211991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.061134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.626212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.578256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.284410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.276983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.173310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.369049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.238371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.168307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.097615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.944607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.925824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.035786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.213931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.171302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.143865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.038501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.085190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.090614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.364715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.497716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.471512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.604794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.455122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.602363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.604072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.438519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.511333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.784588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.736731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.562420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.552790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.466515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.231197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.132542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.949252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.875023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.782212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.845071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.727718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.813652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.808496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.961615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.765697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.730696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.745700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.012183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.125834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.442430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.713215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.599578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.644980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.550419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.494541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.376562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.481536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.597739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.905705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.051057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.173131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.088651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.235046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.431672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.377405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.309535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.225958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.267111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.374561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.281058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.419845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.279382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.442739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.425994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.483661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.373553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.608060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.691566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.842519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.903411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.055766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.369290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.229655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.450261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.497570] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5547.517263] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5547.523309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.740955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.124746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.219581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.040261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.041520] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5553.047276] Lustre: Skipped 130 previous similar messages [ 5553.065492] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.071317] Lustre: Skipped 130 previous similar messages [ 5553.076583] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5553.080464] Lustre: Skipped 130 previous similar messages [ 5553.084699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.091986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5553.095363] Lustre: Skipped 130 previous similar messages [ 5554.204428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.112803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.976344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.797934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.150285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.200821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.209978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.003872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.756193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.662384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.548156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.215266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.947030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.969285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.677953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.491778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.192465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.113493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.953694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.765602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.568665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.270886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.922138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.740717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.478311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.258798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.073122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.935514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.937201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.030252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.089559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.330547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.326209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.219657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.416297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.629275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.535617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.509925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.347500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.128729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.022494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.011469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.111165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.089687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.813633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.688415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.418204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.290732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.088124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.282777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.309688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.328094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.249622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.278485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.106951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.257276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.291642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.405764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.372083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.409063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.648574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.841985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.813968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.929755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.928472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.733659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.473962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.420390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.251504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.206693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.973148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.018575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.981251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.022391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.868851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.062672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.108646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.169566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.206768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.204829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.305501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.338064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.382478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.474239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.299401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.554658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.419980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.296659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.091439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.193272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.111922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.255939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.110669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.107554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.009156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.038066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.784269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.664622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.420044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.340030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.124340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.098709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.053249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.115482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.080865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.158509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.122204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.938042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.897727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.862956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.751620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.777192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.643271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.563564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.505655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.345442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.041004] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.960783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.705621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.553827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.718853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.577776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.729270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.835465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.166742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.165169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.268794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.349729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.205224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.024695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.854457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.816822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.585589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.585876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.387924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.247488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.984065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.765447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.566509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.628924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.619847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.895017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.035248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.942276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.951296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.992823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.755996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.661772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.622368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.889994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.254071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.553884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.889470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.121159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.428134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.483923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.500502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.769319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.785598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.957625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.162175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.230087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.284076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.154470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.187814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.052907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.100965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.022237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.820768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.806937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.706565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.809504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.732752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.853287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.076390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.128577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.964807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.753734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.440725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.162770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.802763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.587274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.559121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.446692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.251623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.194981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.949886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.048338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.100782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.966944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.922863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.742632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.619079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.642932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.738679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.657257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.607657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.361582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.371597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.318775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.468158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.708347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.500121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.607630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.497111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.513138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.402275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.067743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.728038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.540271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.521804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.678533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.576842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.575335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.227533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.513303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.898838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.488771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.708096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.642100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.147076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.508895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.000324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.554427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.876194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.926819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.179063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.070310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.012789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.838900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.761950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.447790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.913553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.865824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.955406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.848723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.016658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.710975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.647766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.486189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.532021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.307212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.214086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.989607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.894701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.825494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.885796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.625483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.360305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.979575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.809501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.545706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.389598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.202172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.027775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.931062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.915497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.830437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.701227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.542344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.572341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.622970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.621955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.395214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.104075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.854593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.779140] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5809.788522] Lustre: Skipped 266 previous similar messages [ 5809.820446] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5809.839035] Lustre: Skipped 266 previous similar messages [ 5809.846207] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5809.858321] Lustre: Skipped 266 previous similar messages [ 5809.869410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.881874] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5809.885015] Lustre: Skipped 266 previous similar messages [ 5810.968497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.864542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.786401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.525755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.260694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.984329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.683464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.585252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.528301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.345594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.069188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.890786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.987701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.893223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.890070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.978554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.967675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.028337] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5826.038586] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5826.039907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.886428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.787475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.722888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.713699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.968077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.165446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.107574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.947701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.572831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.320824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.092073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.316173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.352919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.322637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.453947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.364247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.425748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.360648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.355569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.134937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.007460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.749119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.731583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.469340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.406366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.318244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.062995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.842607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.827222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.742852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.810973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.710500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.619605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.422039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.447454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.360281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.206193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.183613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.383725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.171079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.277327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.129855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.191970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.476155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.596258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.665701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.792504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.972213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.975341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.038513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.045444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.857097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.887594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.855934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.748086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.520637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.361852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.083114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.074669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.103855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.095660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.902649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.831157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.715443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.601551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.309861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.422502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.180546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.940441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.677785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.324526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.268321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.281600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.122903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.774855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.591699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.374575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.335683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.137329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.137842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.917404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.014603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.839744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.885956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.689125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.615814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.596822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.490081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.383236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.359091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.179588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.931081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.637807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.481690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.191046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.081951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.835842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.635512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.405093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.287816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.130456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.988864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.002928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.864279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.725709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.720279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.462061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.343098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.265988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.276587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.241478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.207320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.159748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.067866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.006703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.917589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.040690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.005800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.936100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.738779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.564219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.459387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.181109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.084857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.896475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.750934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.607555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.646336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.599400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.601592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.621601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.534805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.524783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.422907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.159225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.108127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.040951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.045493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.908453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.859824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.583352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.549501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.540030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.640677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.547285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.841285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.082643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.840389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.766116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.647498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.661569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.833745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.963513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.108311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.399698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.715193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.097710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.215988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.613485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.742894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.063930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.525770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.767585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.701466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.754669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.670522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.784162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.780626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.764385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.618912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.674813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.596319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.553922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.799408] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5990.822430] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5990.825657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.077754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.559863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.742309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.892207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.902941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.785133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.685461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.546931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.580047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.440142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.399351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.272126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.166955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.981856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.934520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.784408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.762457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.596718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.729467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.520380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.310912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.177640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.187806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.897971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.080578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.863698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.893829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.791298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.851738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.805350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.679434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.561839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.495840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.387833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.485588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.522901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.287898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.401688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.327135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.380601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.324539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.347194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.143069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.139276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.049615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.391300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.440451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.242644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.093562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.442638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.689918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.799224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.828539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.237505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.423664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.367203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.217796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.220895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.360925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.266885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.210385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.946785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.818260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.745216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.920748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.861099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.793213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.742698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.709945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.781695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.640068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.678482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.799182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.858357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.738200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.085397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.364361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.372549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.258937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.133282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.003134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.975823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.999765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.903968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.909174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.912195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.197789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.169399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.142790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.829101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.750081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.688620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.132532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.432987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.614689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.804544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.747910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.779387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.974727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.419922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.878467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.934592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.154183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.354136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.369838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.408862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.505764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.530345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.637407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.976268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.169873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.480304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.514896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.361261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.337052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.780203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.937812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.707634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.844094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.025255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.897830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.977685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.866855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.084734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.032217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.198488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.378801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.416315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.436454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.286313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.397443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.351476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.602334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.542148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.481793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.314088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.334528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.098536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.286299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.327398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.580173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.549708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.887354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.771660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.934210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.428570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.786982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.081691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.678387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.075147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.146146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.252538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.062909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.950506] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6150.953236] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6150.963818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.758254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.664328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.611659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.966197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.044641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.960550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.165224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.082901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.069381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.960287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.924754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.015329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.295250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.368887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.486709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.421728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.711363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.059598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.413220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.262917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.227327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.478382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.916917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.041515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.899913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.007360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.935820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.891304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.841345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.775756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.623798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.618857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.426615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.198912] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6186.210549] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6186.213554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.215145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.184632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.047955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.047734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.906806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.855920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.719192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.616309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.494208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.775649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.747826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.839834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.860831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.139550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.596995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.960989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.092469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.984656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.084146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.082892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.268095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.948669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.874378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.697311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.076136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.459731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.406993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.656500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.610385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.664693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.626455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.528789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.514234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.652667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.462536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.555366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.382134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.367364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.459889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.729404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.069438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.526341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.555045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.558904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.766890] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6233.781664] LustreError: 172167:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6233.790693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.817283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.148956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.170673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.375302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.577898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.824192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.711085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.033970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.442371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.654511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.996364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.169161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.203923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.057658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.164374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.368560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.135291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.206868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.352719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.236420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.459523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.648763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.561676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.914515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.209814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.029805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.032544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.877586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.696889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.618334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.627915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.639735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.558614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.737906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.842402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.887517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.800320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.008818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.303341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.063190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.748814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.561269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.393069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.243813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.208237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.109783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.936142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.793135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.195937] LustreError: 173972:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6613.210823] LustreError: 172167:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7021.037706] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7021.045828] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7021.052650] Lustre: Skipped 474 previous similar messages [ 7021.069120] Lustre: Skipped 1 previous similar message [ 7025.661970] Lustre: server umount lustre-OST0000 complete [ 7029.231234] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7029.241438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7031.264369] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7035.508722] Lustre: server umount lustre-MDT0000 complete [ 7036.400275] LustreError: 172142:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7036.418491] LustreError: 172142:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7039.086636] LustreError: 172127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739108 with bad export cookie 13024934941726482632 [ 7039.095076] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7039.410421] Lustre: server umount lustre-MDT0001 complete [ 7049.429835] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7051.873227] Key type lgssc unregistered [ 7052.209484] LNet: 227379:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7052.227333] LNetError: 227379:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7052.254726] LNet: Removed LNI 192.168.202.111@tcp [ 7053.167413] Key type .llcrypt unregistered [ 7053.169932] Key type ._llcrypt unregistered [ 7077.547258] Key type ._llcrypt registered [ 7077.553702] Key type .llcrypt registered [ 7077.656075] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7078.769391] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7078.885600] alg: No test for adler32 (adler32-zlib) [ 7079.993940] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 7080.193978] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7081.855147] Key type lgssc registered [ 7083.081881] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7090.448929] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7090.464953] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7092.001888] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7096.902025] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7101.421175] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7101.468398] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7106.544175] LustreError: 228796:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7108.989404] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7109.881485] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7116.466347] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7126.088344] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7132.138506] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7141.652927] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7142.226905] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7150.930937] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7158.389503] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7161.723054] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7168.341125] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7171.008872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7171.023608] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7171.030727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.056505] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7171.056908] LustreError: 228815:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7174.727686] Lustre: lustre-OST0000: Not available for connect from 192.168.202.11@tcp (stopping) [ 7175.137172] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7175.153856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7177.187975] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7177.204990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7177.241262] Lustre: Skipped 2 previous similar messages [ 7179.168355] Lustre: server umount lustre-OST0000 complete [ 7191.135141] LustreError: 228815:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7213.750811] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7214.202259] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7217.253444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7217.270523] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7220.034855] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7226.379790] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7227.500304] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.518093] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7228.073720] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7234.294798] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a2bc925a000.ost_server_uuid 50 [ 7235.727698] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a2bc925a000.ost_server_uuid in FULL state after 0 sec [ 7241.184647] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7241.194415] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7241.207055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7243.235297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7246.092370] Lustre: server umount lustre-OST0000 complete [ 7252.960537] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7252.972482] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7252.984291] Lustre: Skipped 1 previous similar message [ 7252.989852] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7252.992835] Lustre: Skipped 1 previous similar message [ 7255.727336] Lustre: server umount lustre-MDT0000 complete [ 7258.597415] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7258.612250] LustreError: 228795:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7258.802730] LustreError: 228775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739328 with bad export cookie 9186258311587690042 [ 7258.809814] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7258.813663] LustreError: 228775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7259.125504] Lustre: server umount lustre-MDT0001 complete [ 7270.040103] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 7272.758744] Key type lgssc unregistered [ 7273.151203] LNet: 234010:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7273.159582] LNetError: 234010:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7273.180747] LNet: Removed LNI 192.168.202.111@tcp [ 7274.250198] Key type .llcrypt unregistered [ 7274.252274] Key type ._llcrypt unregistered [ 7308.764700] Key type ._llcrypt registered [ 7308.766776] Key type .llcrypt registered [ 7308.861469] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7309.948786] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7310.108597] alg: No test for adler32 (adler32-zlib) [ 7311.118453] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 7311.325881] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 7312.959303] Key type lgssc registered [ 7314.064827] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7326.132620] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7326.145748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7327.773749] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7334.026547] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7337.956361] LustreError: 235742:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7337.987717] LustreError: 235742:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7343.081381] LustreError: 235743:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7343.132434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7343.491936] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7346.902965] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7350.726494] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7357.689567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7358.185947] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7365.609463] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7372.550042] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7375.563817] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7375.718629] Lustre: lustre-OST0001: new disk, initializing [ 7375.722607] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7375.727829] Lustre: 238057:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7375.869641] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7381.430319] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7382.083690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7382.092964] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7382.178045] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7393.068875] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7399.369708] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7413.228050] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7413.249114] Lustre: Skipped 2 previous similar messages [ 7413.254701] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7413.273875] Lustre: Skipped 3 previous similar messages [ 7418.339209] LustreError: 235743:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7418.360432] Lustre: server umount lustre-MDT0000 complete [ 7418.366661] LustreError: 235743:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7422.696694] LustreError: 236477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739492 with bad export cookie 14943836859067114622 [ 7422.710472] LustreError: 236477:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7422.724729] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7423.236505] Lustre: server umount lustre-MDT0001 complete [ 7437.880180] Lustre: server umount lustre-OST0000 complete [ 7439.649553] Lustre: 235006:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786739493/real 1786739493] req@ffff9fdb40950a80 x1873532033282048/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786739509 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7439.689608] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7439.731448] Lustre: Skipped 1 previous similar message [ 7441.742142] Lustre: server umount lustre-OST0001 complete [ 7462.708856] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7472.086386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7472.404118] LustreError: 241161:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7472.534373] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7476.709983] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7477.744120] LustreError: 241162:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7486.377917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7486.909586] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7492.247408] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7495.724305] Lustre: 242304:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7502.810457] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7503.210951] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7510.294866] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7512.441140] LustreError: 242658:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7512.454215] LustreError: 242658:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7518.744792] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 7521.466330] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7529.304506] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7537.691780] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7541.615407] Lustre: 244175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7552.996419] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7553.005111] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7553.009770] Lustre: Skipped 2 previous similar messages [ 7553.013827] Lustre: Skipped 1 previous similar message [ 7558.128388] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7558.142337] Lustre: Skipped 5 previous similar messages [ 7558.649708] Lustre: server umount mylustre-MDT0000 complete [ 7562.612034] LustreError: 241138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739632 with bad export cookie 14943836859067117415 [ 7562.614674] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7562.655969] LustreError: 241138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7562.989089] Lustre: server umount mylustre-MDT0001 complete [ 7577.589753] Lustre: server umount mylustre-OST0000 complete [ 7579.423278] Lustre: 235005:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786739633/real 1786739633] req@ffff9fdb408bf800 x1873532033344000/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786739649 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7579.471568] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7579.483889] Lustre: Skipped 1 previous similar message [ 7582.357190] Lustre: server umount mylustre-OST0001 complete [ 7602.630619] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7613.281367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7613.713467] LustreError: 246482:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7613.732895] LustreError: 246482:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7613.821432] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7613.826666] Lustre: Skipped 1 previous similar message [ 7618.697945] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7631.757831] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7638.109931] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7641.475088] Lustre: 247625:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7649.092242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7656.122664] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7656.770015] LustreError: 247981:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7656.787979] LustreError: 247981:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7659.924041] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 7666.564522] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7672.319924] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 7673.514292] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7680.696700] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7684.565853] Lustre: 249495:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7692.772964] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7692.785599] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7692.792613] Lustre: Skipped 3 previous similar messages [ 7692.804908] Lustre: Skipped 3 previous similar messages [ 7692.994325] Lustre: server umount tfs-MDT0000 complete [ 7696.157289] LustreError: 249497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739765 with bad export cookie 14943836859067119081 [ 7696.169748] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7696.181788] LustreError: 249497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7696.586516] Lustre: server umount tfs-MDT0001 complete [ 7710.576878] Lustre: server umount tfs-OST0000 complete [ 7713.568760] Lustre: 235006:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786739767/real 1786739767] req@ffff9fdb25d70000 x1873532033395072/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786739783 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7713.609244] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7715.027135] Lustre: server umount tfs-OST0001 complete [ 7734.733619] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7745.225177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7745.869520] LustreError: 251745:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7745.882822] LustreError: 251745:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7745.955244] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7745.958993] Lustre: Skipped 3 previous similar messages [ 7750.064827] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7759.934469] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7766.163452] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7769.159601] Lustre: 252887:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7776.801887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7784.745099] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7790.574201] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 7792.878321] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7798.288872] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 7799.893365] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7806.840386] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7810.216472] Lustre: 254757:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7817.827412] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 16:38:05 (1786739885) [ 7834.082097] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7834.088491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7834.108405] Lustre: Skipped 2 previous similar messages [ 7834.138748] Lustre: Skipped 3 previous similar messages [ 7839.545819] Lustre: server umount lustre-MDT0000 complete [ 7843.606652] LustreError: 251721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786739913 with bad export cookie 14943836859067120747 [ 7843.613238] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7843.618105] LustreError: 251721:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7844.332556] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7844.344901] Lustre: Skipped 5 previous similar messages [ 7850.048798] Lustre: server umount lustre-MDT0001 complete [ 7867.871480] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7868.012065] Lustre: server umount lustre-OST0000 complete [ 7886.304507] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7886.487336] Lustre: server umount lustre-OST0001 complete [ 7894.594693] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 7905.873627] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 7939.071696] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7939.071696] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7939.071696] [ 7939.126294] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7939.385685] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7939.424933] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7939.514554] Lustre: lustre-MDT0000: new disk, initializing [ 7939.636117] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7939.638871] Lustre: Skipped 3 previous similar messages [ 7939.659363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7944.543217] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7955.514139] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7955.514139] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7955.514139] [ 7955.563505] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7955.703808] Lustre: 257942:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7955.753559] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7955.760137] Lustre: Skipped 1 previous similar message [ 7955.836835] Lustre: lustre-MDT0001: new disk, initializing [ 7955.980461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7955.996852] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7959.950411] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7970.252572] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7970.496962] Lustre: lustre-OST0000: new disk, initializing [ 7970.500803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7970.507776] Lustre: 259524:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7972.517274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7972.538578] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7972.768985] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7976.079382] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 7986.118389] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7986.364392] Lustre: lustre-OST0001: new disk, initializing [ 7986.367584] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7986.374661] Lustre: 260396:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7988.474065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7988.502086] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7988.705098] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7994.402381] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8007.975401] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 16:41:15 (1786740075) [ 8023.991276] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 16:41:31 (1786740091) [ 8044.513696] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8044.520088] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8044.523630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8044.528649] Lustre: Skipped 4 previous similar messages [ 8044.538683] Lustre: Skipped 3 previous similar messages [ 8046.636899] Lustre: server umount lustre-OST0000 complete [ 8050.150514] LustreError: 259510:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8050.179798] LustreError: 259510:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 8056.369212] Lustre: server umount lustre-MDT0000 complete [ 8060.144255] LustreError: 258737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786740129 with bad export cookie 14943836859067122294 [ 8060.150246] LustreError: 258737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8060.151580] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8066.574241] Lustre: server umount lustre-MDT0001 complete [ 8077.777692] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8088.029431] Lustre: server umount lustre-OST0001 complete [ 8094.030040] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8109.031506] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8147.288256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8147.485922] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8147.509510] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8147.572111] Lustre: lustre-MDT0000: new disk, initializing [ 8147.669765] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8152.058816] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8162.989712] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8163.141317] Lustre: 264954:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8163.716901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8163.728504] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8170.223794] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8181.117188] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8186.660714] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8192.854110] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8193.117448] Lustre: lustre-OST0000: new disk, initializing [ 8193.120137] Lustre: Skipped 1 previous similar message [ 8193.123351] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8193.127565] Lustre: Skipped 2 previous similar messages [ 8193.132124] Lustre: 266927:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8194.534227] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8194.751820] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8201.832195] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8211.878124] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8220.134062] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8220.140197] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8220.141970] LustreError: Skipped 1 previous similar message [ 8220.148266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8220.150091] Lustre: Skipped 5 previous similar messages [ 8220.167872] Lustre: Skipped 11 previous similar messages [ 8224.306558] Lustre: server umount lustre-OST0000 complete [ 8230.393878] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8235.527324] Lustre: server umount lustre-MDT0000 complete [ 8239.747689] LustreError: 266925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786740309 with bad export cookie 14943836859067123771 [ 8239.751480] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8239.768718] LustreError: 266925:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8240.473109] Lustre: server umount lustre-MDT0001 complete [ 8251.923972] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8254.738802] Key type lgssc unregistered [ 8255.018912] LNet: 268686:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8255.028908] LNetError: 268686:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8255.047726] LNet: Removed LNI 192.168.202.111@tcp [ 8256.031229] Key type .llcrypt unregistered [ 8256.033799] Key type ._llcrypt unregistered [ 8280.226640] Key type ._llcrypt registered [ 8280.230249] Key type .llcrypt registered [ 8280.320306] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8281.813694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8281.867894] alg: No test for adler32 (adler32-zlib) [ 8282.932881] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 8283.169209] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8284.800878] Key type lgssc registered [ 8285.900769] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8292.573985] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8292.588567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8293.960300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8297.508523] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8304.099510] LustreError: 270150:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8304.124462] LustreError: 270150:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8304.845745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8305.198048] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8308.886187] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8315.538063] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8320.634449] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8325.857299] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8326.143948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8331.585391] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8338.141466] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8338.415788] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8339.176543] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8343.521551] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8343.533243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8343.539283] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8343.551781] Lustre: Skipped 1 previous similar message [ 8343.580437] Lustre: Skipped 1 previous similar message [ 8347.004746] Lustre: server umount lustre-OST0000 complete [ 8351.200497] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8351.203455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8351.211283] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8353.769873] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8353.778485] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8357.100806] Lustre: server umount lustre-MDT0000 complete [ 8358.883219] LustreError: 270151:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8358.891322] LustreError: 270151:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8360.819291] LustreError: 272677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786740430 with bad export cookie 10052987265695376449 [ 8360.821276] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8360.842833] LustreError: 272677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8361.337710] Lustre: server umount lustre-MDT0001 complete [ 8373.579933] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8376.573333] Key type lgssc unregistered [ 8376.906587] LNet: 273733:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8376.925873] LNetError: 273733:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8376.950126] LNet: Removed LNI 192.168.202.111@tcp [ 8378.020541] Key type .llcrypt unregistered [ 8378.027630] Key type ._llcrypt unregistered [ 8394.777245] Key type ._llcrypt registered [ 8394.782917] Key type .llcrypt registered [ 8394.888625] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8396.190868] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8396.233082] alg: No test for adler32 (adler32-zlib) [ 8397.507785] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 8397.779682] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8399.465276] Key type lgssc registered [ 8401.050966] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8409.733613] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8409.744722] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8425.248360] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8430.367822] LustreError: 274941:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.111@tcp: failed processing log, type 4: rc = -110 [ 8457.056246] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8462.424679] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8468.566992] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8470.558405] Lustre: server umount lustre-OST0000 complete [ 8484.390471] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 8486.764844] Key type lgssc unregistered [ 8486.987448] LNet: 276572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8486.998426] LNetError: 276572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8487.025870] LNet: Removed LNI 192.168.202.111@tcp [ 8487.662578] Key type .llcrypt unregistered [ 8487.664546] Key type ._llcrypt unregistered [ 8521.158601] Key type ._llcrypt registered [ 8521.160281] Key type .llcrypt registered [ 8521.259991] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8522.435127] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8522.490316] alg: No test for adler32 (adler32-zlib) [ 8523.471771] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 8523.780987] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 8525.471409] Key type lgssc registered [ 8527.281639] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8545.719870] loop0: detected capacity change from 0 to 204800000 [ 8546.037741] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8546.064076] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8546.698380] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8548.263742] loop0: detected capacity change from 0 to 204800000 [ 8548.484761] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8548.506409] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8549.182915] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8550.575782] loop0: detected capacity change from 0 to 204800000 [ 8550.850280] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8550.885628] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8551.044295] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8552.623535] loop0: detected capacity change from 0 to 204800000 [ 8552.956820] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8552.982711] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8553.184421] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8554.435025] loop0: detected capacity change from 0 to 204800000 [ 8554.469385] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8559.345850] loop0: detected capacity change from 0 to 204800000 [ 8559.381564] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8563.824699] loop0: detected capacity change from 0 to 204800000 [ 8563.860205] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8568.349291] loop0: detected capacity change from 0 to 204800000 [ 8568.413065] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8572.163394] loop0: detected capacity change from 0 to 204800000 [ 8572.233598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8572.289641] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8572.477326] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8572.493654] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8572.511136] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8572.533656] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8572.551111] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8572.569436] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8572.578378] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8572.608045] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8572.623450] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8572.635710] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8572.651667] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8572.668543] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8572.685294] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8572.701873] Lustre: 279985:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8578.148135] Lustre: server umount MGS complete [ 8579.351480] loop0: detected capacity change from 0 to 204800000 [ 8579.429640] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8580.076941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8580.140799] Lustre: lustre-MDT0000: Aborting client recovery [ 8581.239745] loop1: detected capacity change from 0 to 204800000 [ 8581.457088] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8581.592354] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8581.608859] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8581.621667] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8581.631293] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8581.640934] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8581.650862] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8581.662583] Lustre: 280419:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8581.675927] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8581.842849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8581.920203] Lustre: lustre-MDT0001: Aborting client recovery [ 8582.858365] loop2: detected capacity change from 0 to 204800000 [ 8583.033513] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8583.177611] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8583.197358] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8583.214479] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8583.236988] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8583.258313] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8583.281615] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8583.291989] Lustre: 280499:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8583.462655] Lustre: Found index 0 for lustre-OST0000, updating log [ 8583.507493] Lustre: 280517:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8583.642109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8583.683994] Lustre: lustre-OST0000: Aborting recovery [ 8584.701376] loop3: detected capacity change from 0 to 204800000 [ 8584.904896] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8585.006764] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8585.011589] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8585.027510] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8585.039811] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8585.047039] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8585.054922] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8585.064736] Lustre: 280582:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8585.081639] Lustre: Found index 1 for lustre-OST0001, updating log [ 8585.108265] Lustre: 280588:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8585.193311] Lustre: lustre-OST0001: Aborting recovery [ 8589.295194] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8589.313399] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8589.341861] LustreError: 280363:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8589.355608] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8589.384608] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8591.412052] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8591.419444] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8594.363896] LustreError: 280330:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1518us: rc = -2 [ 8594.884699] LustreError: 280327:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 477us: rc = -2 [ 8594.902842] LustreError: 280327:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 15 previous similar messages [ 8611.807853] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8611.809801] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8611.838747] Lustre: Skipped 2 previous similar messages [ 8611.849732] Lustre: Skipped 3 previous similar messages [ 8614.326801] Lustre: server umount lustre-MDT0000 complete [ 8615.424893] LustreError: 280314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786740685 with bad export cookie 7056466417525341744 [ 8615.437245] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8616.927897] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8616.929038] LustreError: 280329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8616.929161] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8616.960242] Lustre: Skipped 2 previous similar messages [ 8617.022315] LustreError: 280329:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8620.962224] LustreError: 280327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8620.964637] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8620.982267] LustreError: 280327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8620.997658] Lustre: Skipped 2 previous similar messages [ 8622.097574] Lustre: server umount lustre-MDT0001 complete [ 8629.613650] Lustre: server umount lustre-OST0000 complete [ 8637.297783] Lustre: server umount lustre-OST0001 complete [ 8644.371784] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 16:51:52 (1786740712) [ 8679.452971] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 8696.130761] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 8731.040941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8731.227695] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8731.247653] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8731.314220] Lustre: lustre-MDT0000: new disk, initializing [ 8731.395901] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8731.409354] Lustre: Skipped 1 previous similar message [ 8731.429509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8737.022979] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8748.441230] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8748.586269] Lustre: 284679:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8748.617978] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8748.620375] Lustre: Skipped 1 previous similar message [ 8748.678842] Lustre: lustre-MDT0001: new disk, initializing [ 8748.802037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8748.866675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8748.878339] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8753.230049] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8762.369147] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8768.880920] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8776.313183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8776.639424] Lustre: lustre-OST0000: new disk, initializing [ 8776.647805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8776.653664] Lustre: 286648:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8776.808401] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8782.475992] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8784.547935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8784.574054] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8784.774058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8792.003532] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8797.894861] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8809.109631] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8813.051419] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8818.884370] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8856.563957] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8856.570683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8856.572044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8856.582778] Lustre: Skipped 1 previous similar message [ 8861.691907] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8861.703480] Lustre: Skipped 2 previous similar messages [ 8869.343288] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8869.752541] Lustre: server umount lustre-OST0000 complete [ 8877.034083] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8877.041979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8877.063771] LustreError: Skipped 1 previous similar message [ 8877.097318] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8877.104869] Lustre: Skipped 3 previous similar messages [ 8880.726323] Lustre: server umount lustre-MDT0000 complete [ 8882.148254] LustreError: 284688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8882.176529] LustreError: 284688:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8884.524681] LustreError: 284674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786740954 with bad export cookie 7056466417525344453 [ 8884.535186] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8884.536119] LustreError: 284674:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8884.885590] Lustre: server umount lustre-MDT0001 complete [ 8892.865527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8897.073844] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8901.658699] Lustre: server umount MGS complete [ 8911.125533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8911.434605] LustreError: 290464:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8911.457948] LustreError: 290477:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8911.506984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8915.211889] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8916.960728] LustreError: 290465:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8921.687216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8926.099518] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8932.033443] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8937.742823] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8944.501368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8944.894169] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8944.901731] Lustre: Skipped 1 previous similar message [ 8950.540784] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 8958.526417] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8983.008691] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8983.020618] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8983.030918] Lustre: Skipped 1 previous similar message [ 8983.036143] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8988.415966] Lustre: server umount lustre-OST0000 complete [ 8993.764735] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8993.774107] LustreError: Skipped 1 previous similar message [ 8993.782778] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8993.798881] Lustre: Skipped 1 previous similar message [ 8999.277587] Lustre: server umount lustre-MDT0000 complete [ 8999.394053] LustreError: 290460:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8999.412080] LustreError: 290460:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9002.931067] LustreError: 290445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786741072 with bad export cookie 7056466417525346280 [ 9002.938371] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9002.942398] LustreError: 290445:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9003.234862] Lustre: server umount lustre-MDT0001 complete [ 9016.420298] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9020.500749] Key type lgssc unregistered [ 9021.080565] LNet: 294406:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9021.102077] LNetError: 294406:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9021.151342] LNet: Removed LNI 192.168.202.111@tcp [ 9022.506984] Key type .llcrypt unregistered [ 9022.509604] Key type ._llcrypt unregistered [ 9068.674320] Key type ._llcrypt registered [ 9068.675891] Key type .llcrypt registered [ 9068.796204] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9083.658250] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9084.718747] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9084.878281] alg: No test for adler32 (adler32-zlib) [ 9085.906029] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 9086.090806] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9087.735198] Key type lgssc registered [ 9088.607436] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9119.764846] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9119.828164] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9121.095967] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9121.133422] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9121.214486] Lustre: lustre-MDT0000: new disk, initializing [ 9121.286668] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9121.316833] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9124.756329] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9135.542370] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9135.652078] Lustre: 298368:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9135.753542] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9135.756795] Lustre: Skipped 1 previous similar message [ 9135.848922] Lustre: lustre-MDT0001: new disk, initializing [ 9135.985439] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9136.027485] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9136.034134] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9139.549434] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9149.145067] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9155.081113] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9162.043488] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9162.343846] Lustre: lustre-OST0000: new disk, initializing [ 9162.346530] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9162.351922] Lustre: 300334:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9162.458912] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9168.512462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9168.523722] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9168.711645] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9169.132237] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9179.144150] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9186.579288] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9200.720416] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9204.077305] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9209.925069] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9245.665411] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9245.669132] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9245.688401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9245.705020] Lustre: Skipped 1 previous similar message [ 9250.240831] Lustre: server umount lustre-OST0000 complete [ 9253.856692] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9253.871080] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9253.882630] Lustre: Skipped 1 previous similar message [ 9253.887457] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9255.907683] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9255.930954] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9261.026997] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9261.042278] Lustre: Skipped 2 previous similar messages [ 9266.148985] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9266.173900] Lustre: Skipped 1 previous similar message [ 9268.191357] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9268.838034] Lustre: server umount lustre-MDT0000 complete [ 9273.013626] LustreError: 301841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786741342 with bad export cookie 16569460736355205662 [ 9273.020190] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9273.027341] LustreError: 301841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9273.329787] Lustre: server umount lustre-MDT0001 complete [ 9281.847945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9286.354410] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9290.953485] Lustre: server umount MGS complete [ 9303.757891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9304.111068] LustreError: 304262:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9304.126429] LustreError: 304262:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9304.146577] LustreError: 304275:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9304.241367] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9307.994605] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9309.666595] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9313.766781] LustreError: 304262:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9318.768450] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9318.896884] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9319.341679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9326.493418] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9340.113916] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9349.842387] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9358.904921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9359.520733] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9365.634537] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9373.745723] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9404.898808] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9404.904407] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9404.944352] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9404.951607] Lustre: Skipped 1 previous similar message [ 9409.721788] Lustre: server umount lustre-OST0000 complete [ 9415.137260] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9415.149884] Lustre: Skipped 1 previous similar message [ 9420.131361] Lustre: server umount lustre-MDT0000 complete [ 9420.262993] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9420.284117] LustreError: 304263:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9423.811335] LustreError: 304242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786741493 with bad export cookie 16569460736355207503 [ 9423.823313] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9423.833578] LustreError: 304242:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9424.222848] Lustre: server umount lustre-MDT0001 complete [ 9436.635203] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [ 9440.086212] Key type lgssc unregistered [ 9440.417675] LNet: 308211:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9440.423400] LNetError: 308211:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9440.441515] LNet: Removed LNI 192.168.202.111@tcp [ 9441.466884] Key type .llcrypt unregistered [ 9441.470627] Key type ._llcrypt unregistered [ 9464.499578] Key type ._llcrypt registered [ 9464.502178] Key type .llcrypt registered [ 9464.611426] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [ 9482.843333] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9483.812298] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9484.161586] alg: No test for adler32 (adler32-zlib) [ 9485.260908] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [ 9485.503762] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [ 9487.168055] Key type lgssc registered [ 9488.294445] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9515.362527] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [ 9532.441848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9532.500202] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9533.837992] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9533.873860] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9533.965254] Lustre: lustre-MDT0000: new disk, initializing [ 9534.079610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9534.106383] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9540.514572] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9550.883450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9551.149730] Lustre: lustre-OST0000: new disk, initializing [ 9551.153805] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9551.158765] Lustre: Skipped 1 previous similar message [ 9551.170167] Lustre: 312264:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9551.309328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9557.682413] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [ 9559.709247] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9559.715789] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9559.738488] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10361.534020] Lustre: server umount lustre-MDT0000 complete [10367.555914] LustreError: 311449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786742437 with bad export cookie 11931742234610936959 [10367.567619] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10377.387846] Lustre: server umount lustre-OST0000 complete [10405.914073] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [10417.536881] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10455.565406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10455.846912] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10455.887963] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10456.013497] Lustre: lustre-MDT0000: new disk, initializing [10456.151461] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10456.175225] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10459.728982] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10470.334244] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10470.452500] Lustre: 316595:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10470.522453] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10470.526082] Lustre: Skipped 1 previous similar message [10470.711101] Lustre: lustre-MDT0001: new disk, initializing [10470.883144] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10470.968234] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10470.994729] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10475.627139] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10485.167180] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10490.780503] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10497.061625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10497.298482] Lustre: lustre-OST0000: new disk, initializing [10497.304609] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10497.311447] Lustre: 318561:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10497.407131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10504.120400] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10506.864606] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10506.873912] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10507.024545] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10513.031630] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10518.052679] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10518.207253] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10521.998762] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10522.311760] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10527.715878] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10527.725909] Lustre: Skipped 1 previous similar message [10527.727804] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10527.728537] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10527.743845] Lustre: Skipped 1 previous similar message [10530.852923] Lustre: server umount lustre-OST0000 complete [10537.442783] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10537.449687] LustreError: Skipped 1 previous similar message [10537.453587] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10537.487529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10541.683388] Lustre: server umount lustre-MDT0000 complete [10545.540610] LustreError: 316587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786742615 with bad export cookie 11931742234613877778 [10545.565570] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10545.573257] LustreError: 316587:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10546.047468] Lustre: server umount lustre-MDT0001 complete [10550.159837] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10551.838941] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 17:23:40 (1786742620) [10560.162159] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10560.432298] LustreError: 320876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10560.444846] LustreError: 320876:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10560.489083] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10565.618720] LustreError: 320877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10565.625487] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10570.729165] LustreError: 320876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10575.083281] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10575.448959] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10580.667154] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10589.355346] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10596.871502] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10605.344624] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10605.723917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10612.064825] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10619.968677] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10626.717754] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10626.950483] Lustre: lustre-OST0001: new disk, initializing [10626.956376] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10626.965088] Lustre: 323712:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10627.086821] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10633.728674] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10636.532105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10636.575913] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10636.665618] LustreError: 324181:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10637.737331] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10637.749181] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10638.820568] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10638.857629] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10640.870386] LustreError: 324181:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10640.883960] LustreError: 324181:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10645.015947] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10645.101355] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10645.116532] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10645.458440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10653.128574] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a2be09f2800.ost_server_uuid 50 [10653.348897] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10653.361703] LustreError: 324190:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10655.175917] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a2be09f2800.ost_server_uuid in FULL state after 0 sec [10665.752357] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8a2be09f2800.ost_server_uuid 50 [10669.165528] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8a2be09f2800.ost_server_uuid in IDLE state after 0 sec [10669.877378] LustreError: 324181:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10669.901653] LustreError: 324181:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [10672.189335] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10687.456455] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10687.458959] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10687.463859] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10687.467194] Lustre: Skipped 2 previous similar messages [10687.477759] Lustre: Skipped 2 previous similar messages [10690.601335] Lustre: lustre-OST0001: Not available for connect from 192.168.202.11@tcp (stopping) [10692.272165] Lustre: server umount lustre-OST0001 complete [10693.102128] LustreError: 322697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10693.150678] LustreError: 322697:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10698.215731] LustreError: 322699:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10698.228595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10698.229168] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10698.229659] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10698.262864] LustreError: 322699:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10698.294019] LustreError: Skipped 1 previous similar message [10703.883867] Lustre: server umount lustre-OST0000 complete [10713.580532] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10713.585974] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10713.616322] Lustre: Skipped 1 previous similar message [10713.645828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10713.659970] Lustre: Skipped 4 previous similar messages [10716.006951] Lustre: server umount lustre-MDT0000 complete [10718.728163] LustreError: 320877:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10718.745127] LustreError: 320877:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10721.111757] LustreError: 320856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786742790 with bad export cookie 11931742234613878660 [10721.134796] LustreError: 320856:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10721.135467] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10721.640531] Lustre: server umount lustre-MDT0001 complete [10735.619517] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [10739.742656] Key type lgssc unregistered [10740.041418] LNet: 326405:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10740.053605] LNetError: 326405:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10740.103898] LNet: Removed LNI 192.168.202.111@tcp [10741.147740] Key type .llcrypt unregistered [10741.150979] Key type ._llcrypt unregistered [10770.224249] Key type ._llcrypt registered [10770.229179] Key type .llcrypt registered [10770.383367] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [10771.618653] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10771.677061] alg: No test for adler32 (adler32-zlib) [10772.725180] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [10773.160413] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [10774.887391] Key type lgssc registered [10776.528848] Lustre: Echo OBD driver; http://www.lustre.org/ [10787.502407] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10787.515716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10788.992795] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10793.715584] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10798.574861] LustreError: 327827:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10798.592697] LustreError: 327827:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10802.837846] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10803.684893] LustreError: 327828:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10808.804084] LustreError: 327827:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10809.565622] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10809.918238] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10814.383469] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10823.323529] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10823.753742] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10829.353933] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10833.963293] LustreError: 329510:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10834.004978] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10837.000116] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10840.035590] LustreError: 329507:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10840.075076] LustreError: 329507:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10843.753061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10844.082304] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10849.294318] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10851.057658] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [10859.873896] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10865.554758] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10865.667296] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10972.154642] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10972.170216] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10972.171911] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10972.180496] Lustre: Skipped 1 previous similar message [10975.756244] Lustre: server umount lustre-OST0001 complete [10977.259485] LustreError: 330074:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10977.281708] LustreError: 330074:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10982.369175] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10982.374517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10982.389299] Lustre: Skipped 2 previous similar messages [10982.416963] Lustre: Skipped 1 previous similar message [10987.489947] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10987.937839] Lustre: server umount lustre-OST0000 complete [10992.611870] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10992.618799] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10992.627501] Lustre: Skipped 1 previous similar message [10992.643149] Lustre: Skipped 2 previous similar messages [10997.732382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10997.735220] Lustre: Skipped 1 previous similar message [10998.681709] Lustre: server umount lustre-MDT0000 complete [11002.330647] LustreError: 327807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786743072 with bad export cookie 3621179296313866451 [11002.336602] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11002.339755] LustreError: 327807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11002.714445] Lustre: server umount lustre-MDT0001 complete [11013.237668] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11015.681959] Key type lgssc unregistered [11016.007932] LNet: 332771:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11016.025471] LNetError: 332771:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11016.042190] LNet: Removed LNI 192.168.202.111@tcp [11016.992793] Key type .llcrypt unregistered [11016.994636] Key type ._llcrypt unregistered [11060.996193] Key type ._llcrypt registered [11060.998391] Key type .llcrypt registered [11061.080680] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11062.050557] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11062.113932] alg: No test for adler32 (adler32-zlib) [11063.104162] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [11063.413046] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11065.111175] Key type lgssc registered [11066.321382] Lustre: Echo OBD driver; http://www.lustre.org/ [11075.364775] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11075.388950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11076.997490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11082.312580] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11086.313201] LustreError: 335062:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11086.330097] LustreError: 335062:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11091.427662] LustreError: 335063:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11092.501920] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11092.903544] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11096.885796] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11100.572510] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11106.916374] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11107.415766] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11113.110896] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11117.611027] LustreError: 336551:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11122.404421] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11122.640922] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11123.705934] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11123.723109] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11127.461721] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11138.916880] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11147.165705] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11159.522350] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11159.526691] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11159.533241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11159.533485] Lustre: Skipped 3 previous similar messages [11159.541235] Lustre: Skipped 3 previous similar messages [11164.642506] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11164.651794] Lustre: Skipped 1 previous similar message [11165.095786] Lustre: server umount lustre-MDT0000 complete [11169.693725] LustreError: 335043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786743239 with bad export cookie 11518757381328317249 [11169.699939] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11169.749948] LustreError: 335043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11169.803975] LustreError: 335057:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11169.808097] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11169.816344] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11169.835485] LustreError: 335057:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11169.906548] Lustre: Skipped 1 previous similar message [11170.393832] Lustre: server umount lustre-MDT0001 complete [11177.632672] Lustre: server umount lustre-OST0000 complete [11182.026910] Lustre: server umount lustre-OST0001 complete [11206.518414] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11219.407867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11220.301912] LustreError: 340341:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11220.538499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11227.590893] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11230.700253] LustreError: 340342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11230.733377] LustreError: 340342:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11238.249561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11238.573875] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11243.407871] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11246.236942] Lustre: 341481:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11252.833977] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11259.161907] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11263.543074] LustreError: 341832:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11263.567778] LustreError: 341832:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11265.697205] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11269.394755] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11269.651777] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11269.656216] Lustre: Skipped 1 previous similar message [11274.765439] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11277.108638] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11286.754921] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11290.352352] Lustre: 343352:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11294.690576] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11294.695944] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11294.724662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11294.734644] Lustre: Skipped 1 previous similar message [11299.579774] Lustre: server umount lustre-MDT0000 complete [11300.322445] LustreError: 340342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11300.363301] LustreError: 340342:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11303.147962] LustreError: 340321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786743372 with bad export cookie 11518757381328318950 [11303.155578] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11303.156060] LustreError: 340321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11303.480351] Lustre: server umount lustre-MDT0001 complete [11317.618509] Lustre: server umount lustre-OST0000 complete [11321.824244] Lustre: 334326:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786743375/real 1786743375] req@ffff9fdb41677800 x1873535967540224/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786743391 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11321.859779] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11321.890532] Lustre: Skipped 3 previous similar messages [11323.212048] Lustre: server umount lustre-OST0001 complete [11329.176286] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11331.263669] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11334.581097] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 17:36:41 (1786743401) [11360.733854] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11371.513671] loop0: detected capacity change from 0 to 1099511627776 [11372.650580] SGI XFS with ACLs, security attributes, quota, no debug enabled [11372.657575] *** VALIDATE xfs *** [11372.680416] XFS (loop0): Mounting V5 Filesystem [11372.710866] XFS (loop0): Ending clean mount [11374.349430] loop1: detected capacity change from 0 to 18691697672192 [11481.556994] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [11492.942263] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11534.608839] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11534.998589] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11535.030480] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11535.120981] Lustre: lustre-MDT0000: new disk, initializing [11535.250064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11535.286135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11540.398641] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11550.838512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11550.962200] Lustre: 348488:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11550.999852] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11551.002852] Lustre: Skipped 1 previous similar message [11551.050314] Lustre: lustre-MDT0001: new disk, initializing [11551.171112] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11551.195845] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11555.057746] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11565.662881] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11572.642408] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11581.737483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11581.914800] Lustre: lustre-OST0000: new disk, initializing [11581.916850] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11581.920798] Lustre: 350459:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11583.204355] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11583.219456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11583.347210] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11589.934098] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11601.686248] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11607.843896] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11608.342463] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11613.577309] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11613.885202] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11619.301952] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11619.312175] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11619.322659] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11619.331741] Lustre: Skipped 1 previous similar message [11619.350050] Lustre: Skipped 5 previous similar messages [11622.723581] Lustre: server umount lustre-OST0000 complete [11628.001664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11628.038858] LustreError: Skipped 1 previous similar message [11628.057939] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11628.101682] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11632.789575] Lustre: server umount lustre-MDT0000 complete [11634.657278] LustreError: 348495:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11634.681133] LustreError: 348495:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11638.280991] LustreError: 348482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786743708 with bad export cookie 11518757381328320497 [11638.303043] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11638.305306] LustreError: 348482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11639.209808] Lustre: server umount lustre-MDT0001 complete [11650.588058] XFS (loop0): Unmounting Filesystem [11657.919290] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 17:42:04 (1786743724) [11665.512218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11666.090578] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11666.098714] Lustre: Skipped 2 previous similar messages [11669.997025] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11678.372714] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11682.675770] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11688.447496] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11694.186917] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11701.533467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11710.419654] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11721.024485] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11731.936794] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11731.939738] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11731.944154] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11731.970721] Lustre: Skipped 2 previous similar messages [11731.989077] Lustre: Skipped 3 previous similar messages [11736.472702] Lustre: server umount lustre-OST0000 complete [11748.044752] Lustre: server umount lustre-MDT0000 complete [11752.942681] LustreError: 353000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786743822 with bad export cookie 11518757381328321379 [11752.945665] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11752.968029] LustreError: 353000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11753.588833] Lustre: server umount lustre-MDT0001 complete [11768.152554] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [11771.892065] Key type lgssc unregistered [11772.266920] LNet: 356650:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11772.275871] LNetError: 356650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11772.293460] LNet: Removed LNI 192.168.202.111@tcp [11773.391133] Key type .llcrypt unregistered [11773.398293] Key type ._llcrypt unregistered [11813.120701] Key type ._llcrypt registered [11813.127394] Key type .llcrypt registered [11813.283730] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [11816.062374] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11816.166794] alg: No test for adler32 (adler32-zlib) [11817.568889] Lustre: Lustre: Build Version: 2.17.56_2_gd9f03f2 [11818.015931] LNet: Added LNI 192.168.202.111@tcp [8/256/0/180] [11819.807189] Key type lgssc registered [11821.129167] Lustre: Echo OBD driver; http://www.lustre.org/ [11830.737876] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11830.751949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11832.118540] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11839.420207] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11841.505406] LustreError: 358224:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11841.527810] LustreError: 358224:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11846.624816] LustreError: 358223:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11847.670660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11847.961295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11852.901414] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11860.057995] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11865.537189] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11872.464502] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11872.680227] LustreError: 360044:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11872.847396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11878.854924] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11885.630833] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11893.811552] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [11894.243637] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11894.251454] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11897.827797] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11897.832098] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11897.833109] Lustre: Skipped 1 previous similar message [11898.918342] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [11898.925657] Lustre: Skipped 1 previous similar message [11899.691644] Lustre: server umount lustre-MDT0001 complete [11902.947078] LustreError: 358217:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11902.978305] LustreError: 358217:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11908.092056] LustreError: 358219:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11908.113775] LustreError: 358219:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11911.226883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11911.315700] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11911.319971] Lustre: Found index 1 for lustre-MDT0001, updating log [11911.362529] Lustre: 358213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11911.742294] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11915.573370] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11916.787129] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11917.321375] LustreError: 361759:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11917.338530] LustreError: 361759:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11917.375665] LustreError: 358216:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11917.472793] LustreError: 361773:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [11917.487142] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11917.487142] [11924.579203] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.11@tcp (stopping) [11927.009091] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11927.029034] Lustre: Skipped 1 previous similar message [11928.757790] Lustre: server umount lustre-MDT0001 complete [11929.613341] LustreError: 361015:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.11@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11929.650895] LustreError: 361015:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11938.437228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11938.566547] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11938.571987] Lustre: Found index 1 for lustre-MDT0001, updating log [11938.622261] Lustre: 358213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11939.052864] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11944.083723] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [11944.427287] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11944.466262] LustreError: 362757:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11944.477577] LustreError: 362757:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11944.493119] LustreError: 358216:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11944.508322] LustreError: 358216:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11944.613638] LustreError: 362768:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11944.628490] LustreError: 362768:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11944.639303] LustreError: 362768:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -17 [11944.653997] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.111@tcp [11944.653997] [12251.627647] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12251.643372] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12251.643648] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12251.650952] Lustre: Skipped 2 previous similar messages [12256.413561] Lustre: server umount lustre-MDT0001 complete [12256.738199] LustreError: 358223:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12256.751432] LustreError: 358223:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [12266.905650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12267.005380] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12267.011665] Lustre: Found index 1 for lustre-MDT0001, updating log [12267.040533] Lustre: 358213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12267.394601] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12272.010393] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12272.616677] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12274.164055] LustreError: 371343:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12274.171031] LustreError: 371343:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12274.185240] LustreError: 358216:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12274.193103] LustreError: 358216:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12274.249478] LustreError: 371351:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12274.263155] LustreError: 371351:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.111@tcp: cfg command failed: rc = -114 [12274.269537] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12274.269537] [12593.122417] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12593.129778] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12593.131509] Lustre: Skipped 2 previous similar messages [12593.148364] Lustre: Skipped 3 previous similar messages [12605.407177] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12605.559464] Lustre: server umount lustre-MDT0000 complete [12608.486384] LustreError: 358220:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12608.516449] LustreError: 358220:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [12609.177681] LustreError: 358203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744679 with bad export cookie 14994138791269247023 [12609.185179] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12609.191320] LustreError: 358203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12613.601750] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12613.618282] Lustre: Skipped 1 previous similar message [12613.624159] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12613.631844] Lustre: Skipped 6 previous similar messages [12615.756818] Lustre: server umount lustre-MDT0001 complete [12619.065409] Lustre: server umount lustre-OST0000 complete [12628.288396] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [12638.828207] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12679.810625] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 17:59:08 (1786744748) [12687.690737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12687.944770] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12687.966223] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12688.034595] Lustre: lustre-MDT0000: new disk, initializing [12688.123286] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12688.140702] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12693.139626] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12705.432988] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12705.575988] Lustre: 382002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12705.705912] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12705.710968] Lustre: Skipped 1 previous similar message [12705.874174] Lustre: lustre-MDT0001: new disk, initializing [12705.998854] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12706.042959] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12706.056849] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12710.052968] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12719.878534] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12724.768723] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12731.564952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12731.797634] Lustre: lustre-OST0000: new disk, initializing [12731.801194] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12731.806673] Lustre: 383971:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12731.894064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12738.468544] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_default_debug -1 all [12741.175556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12741.189517] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12741.340888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12748.268399] Lustre: DEBUG MARKER: oleg211-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12756.964335] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12756.969553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12756.999817] Lustre: Skipped 1 previous similar message [12757.007750] Lustre: Skipped 1 previous similar message [12757.650899] LustreError: 384777:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (00000000f6a644aa) refcount nonzero (2) after lock cleanup; forcing cleanup. [12757.903960] Lustre: server umount lustre-MDT0000 complete [12762.012751] LustreError: 381993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744831 with bad export cookie 14994138791269248976 [12762.020941] LustreError: MGC192.168.202.111@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12762.026605] LustreError: 381993:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [12762.026990] LustreError: 382809:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12762.027210] LustreError: 382809:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000aeb1366a x1873536758355456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [12762.081793] LustreError: 382008:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12762.106557] LustreError: 382008:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12762.694882] Lustre: server umount lustre-MDT0001 complete [12768.063079] Lustre: server umount lustre-OST0000 complete [12802.079637] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing set_hostid [12814.381102] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing load_modules_local [12848.249211] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12586 sec ========= 18:01:55 (1786744915) [12850.548572] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 18:01:58 (1786744918) === [12854.177845] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 18:02:02 (1786744922) === [12893.576513] Lustre: DEBUG MARKER: oleg211-server.virtnet: executing unload_modules_local [12897.004410] Key type lgssc unregistered [12897.313816] LNet: 390349:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12897.320302] LNetError: 390349:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12897.334812] LNet: Removed LNI 192.168.202.111@tcp [12898.246449] Key type .llcrypt unregistered [12898.250488] Key type ._llcrypt unregistered