[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-8.fc42 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 982806149 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE2439 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE22D5 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 002295 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE2349 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE23D9 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2411 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe22d5-0xbffe2348] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe22d4] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe2349-0xbffe23d8] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe23d9-0xbffe2410] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2411-0xbffe2438] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001018] APIC: Switch to symmetric I/O mode setup [ 0.003100] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005022] kvm-guest: setup PV IPIs [ 0.007828] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009019] pid_max: default: 32768 minimum: 301 [ 0.010124] LSM: Security Framework initializing [ 0.012073] Yama: becoming mindful. [ 0.013039] SELinux: Initializing. [ 0.015061] *** VALIDATE selinux *** [ 0.021083] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026414] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028021] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029100] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030129] *** VALIDATE tmpfs *** [ 0.032399] *** VALIDATE proc *** [ 0.033210] *** VALIDATE cgroup *** [ 0.034017] *** VALIDATE cgroup2 *** [ 0.035263] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037152] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038006] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039028] Spectre V2 : User space: Vulnerable [ 0.040007] Speculative Store Bypass: Vulnerable [ 0.043329] debug: unmapping init [mem 0xffffffff99e59000-0xffffffff99e60fff] [ 0.045318] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047700] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048036] ... version: 2 [ 0.049020] ... bit width: 48 [ 0.050011] ... generic registers: 4 [ 0.051012] ... value mask: 0000ffffffffffff [ 0.052010] ... max period: 00007fffffffffff [ 0.053016] ... fixed-purpose events: 3 [ 0.054010] ... event mask: 000000070000000f [ 0.055289] rcu: Hierarchical SRCU implementation. [ 0.058060] smp: Bringing up secondary CPUs ... [ 0.059904] x86: Booting SMP configuration: [ 0.060081] .... node #0, CPUs: #1 #2 #3 [ 0.160321] smp: Brought up 1 node, 4 CPUs [ 0.162040] smpboot: Max logical packages: 1 [ 0.163025] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.350466] node 0 deferred pages initialised in 31ms [ 0.360000] devtmpfs: initialized [ 0.360000] x86/mm: Memory block size: 128MB [ 0.465453] gcov: version magic: 0x41383552 [ 0.471000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.471164] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.474471] pinctrl core: initialized pinctrl subsystem [ 0.475000] [ 0.475000] ************************************************************* [ 0.475016] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.478010] ** ** [ 0.480011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.481000] ** ** [ 0.481000] ** This means that this kernel is built to expose internal ** [ 0.481000] ** IOMMU data structures, which may compromise security on ** [ 0.484011] ** your system. ** [ 0.486009] ** ** [ 0.488015] ** If you see this message and you are not debugging the ** [ 0.490019] ** kernel, report this immediately to your vendor! ** [ 0.491000] ** ** [ 0.491023] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.492000] ************************************************************* [ 0.493000] NET: Registered protocol family 16 [ 0.494774] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.495150] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.496000] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.502036] cpuidle: using governor menu [ 0.503882] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.504454] PCI: Using configuration type 1 for base access [ 0.505219] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.514000] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.515033] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.526095] cryptd: max_cpu_qlen set to 1000 [ 0.530405] ACPI: Added _OSI(Module Device) [ 0.531000] ACPI: Added _OSI(Processor Device) [ 0.533037] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.536017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.545544] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.557532] ACPI: Interpreter enabled [ 0.558086] ACPI: PM: (supports S0 S3 S4 S5) [ 0.559012] ACPI: Using IOAPIC for interrupt routing [ 0.560112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.565408] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.591509] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.595069] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.596030] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.604083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.609832] acpiphp: Slot [2] registered [ 0.612150] acpiphp: Slot [5] registered [ 0.613219] acpiphp: Slot [6] registered [ 0.615215] acpiphp: Slot [7] registered [ 0.616113] acpiphp: Slot [8] registered [ 0.618077] acpiphp: Slot [9] registered [ 0.619095] acpiphp: Slot [10] registered [ 0.620089] acpiphp: Slot [3] registered [ 0.621072] acpiphp: Slot [4] registered [ 0.623059] acpiphp: Slot [11] registered [ 0.624144] acpiphp: Slot [12] registered [ 0.625068] acpiphp: Slot [13] registered [ 0.626059] acpiphp: Slot [14] registered [ 0.628092] acpiphp: Slot [15] registered [ 0.629155] acpiphp: Slot [16] registered [ 0.630147] acpiphp: Slot [17] registered [ 0.632085] acpiphp: Slot [18] registered [ 0.634070] acpiphp: Slot [19] registered [ 0.635074] acpiphp: Slot [20] registered [ 0.637065] acpiphp: Slot [21] registered [ 0.638085] acpiphp: Slot [22] registered [ 0.639063] acpiphp: Slot [23] registered [ 0.640159] acpiphp: Slot [24] registered [ 0.642077] acpiphp: Slot [25] registered [ 0.644082] acpiphp: Slot [26] registered [ 0.645113] acpiphp: Slot [27] registered [ 0.646108] acpiphp: Slot [28] registered [ 0.648080] acpiphp: Slot [29] registered [ 0.649088] acpiphp: Slot [30] registered [ 0.650121] acpiphp: Slot [31] registered [ 0.652066] PCI host bridge to bus 0000:00 [ 0.653017] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.654020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.656013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.659025] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.662032] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.664016] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.666168] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.667934] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.671160] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.681022] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.685000] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.687036] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.688034] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.693037] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.705144] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.726000] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.731046] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.732000] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.739014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.755019] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.765744] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.782221] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.813032] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.828017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.864018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 1.086276] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 1.102023] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 1.123020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 1.155023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 1.221251] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 1.235019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 1.275023] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 1.306018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 1.335105] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 1.346014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.366029] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.398012] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 1.411997] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.427017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.446017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.474019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.484615] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.491012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.501017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.519015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.530437] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.533362] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.535433] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.538542] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.540212] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.545083] iommu: Default domain type: Passthrough [ 1.546298] SCSI subsystem initialized [ 1.548111] ACPI: bus type USB registered [ 1.550110] usbcore: registered new interface driver usbfs [ 1.552174] usbcore: registered new interface driver hub [ 1.554075] usbcore: registered new device driver usb [ 1.556125] pps_core: LinuxPPS API ver. 1 registered [ 1.559013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.562060] PTP clock support registered [ 1.564050] EDAC MC: Ver: 3.0.0 [ 1.565000] PCI: Using ACPI for IRQ routing [ 1.566000] NetLabel: Initializing [ 1.567011] NetLabel: domain hash size = 128 [ 1.569009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.572118] NetLabel: unlabeled traffic allowed by default [ 1.574274] vgaarb: loaded [ 1.575235] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.576000] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.584659] clocksource: Switched to clocksource kvm-clock [ 2.160379] VFS: Disk quotas dquot_6.6.0 [ 2.168489] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 2.180981] *** VALIDATE ramfs *** [ 2.185391] *** VALIDATE hugetlbfs *** [ 2.192132] pnp: PnP ACPI init [ 2.205904] pnp: PnP ACPI: found 6 devices [ 2.264859] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 2.278126] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 2.284175] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 2.298483] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 2.325512] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 2.345827] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 2.364550] NET: Registered protocol family 2 [ 2.384447] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 2.452262] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 2.508043] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 2.520176] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 2.535050] TCP: Hash tables configured (established 65536 bind 65536) [ 2.544123] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 2.552935] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.556625] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 2.564217] NET: Registered protocol family 1 [ 2.650547] RPC: Registered named UNIX socket transport module. [ 2.657931] RPC: Registered udp transport module. [ 2.666469] RPC: Registered tcp transport module. [ 2.668612] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 2.673945] NET: Registered protocol family 44 [ 2.677620] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 2.683416] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 2.687103] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 2.694929] PCI: CLS 0 bytes, default 64 [ 2.696839] Unpacking initramfs... [ 7.221863] debug: unmapping init [mem 0xffff9916fcc54000-0xffff9916fffbffff] [ 7.226560] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 7.229519] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 7.232825] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 9.142097] hrtimer: interrupt took 3693164 ns [ 10.587542] Initialise system trusted keyrings [ 10.589265] Key type blacklist registered [ 10.591378] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 10.601738] zbud: loaded [ 10.605793] *** VALIDATE nfs *** [ 10.607568] *** VALIDATE nfs4 *** [ 10.609488] pstore: using deflate compression [ 10.614388] Platform Keyring initialized [ 10.916405] NET: Registered protocol family 38 [ 10.918918] Key type asymmetric registered [ 10.922716] Asymmetric key parser 'x509' registered [ 10.924926] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 10.930356] io scheduler mq-deadline registered [ 10.932237] io scheduler kyber registered [ 10.933481] io scheduler bfq registered [ 10.934948] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 10.937526] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 10.940421] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 10.946271] ACPI: Power Button [PWRF] [ 10.956729] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 10.994795] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 11.026632] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 11.118377] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 11.151958] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 11.180622] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 11.209949] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 11.215340] Non-volatile memory driver v1.3 [ 11.217273] Linux agpgart interface v0.103 [ 11.341260] virtio_blk virtio1: [vda] 134152 512-byte logical blocks (68.7 MB/65.5 MiB) [ 11.344274] vda: detected capacity change from 0 to 68685824 [ 11.367777] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 11.412061] vdb: detected capacity change from 0 to 1073741824 [ 11.517741] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 11.524200] vdc: detected capacity change from 0 to 2621440000 [ 11.800724] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 11.803298] vdd: detected capacity change from 0 to 2621440000 [ 11.820368] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 11.822976] vde: detected capacity change from 0 to 4294967296 [ 11.839319] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 11.842303] vdf: detected capacity change from 0 to 4294967296 [ 11.863384] libphy: Fixed MDIO Bus: probed [ 11.889908] usbcore: registered new interface driver usbserial_generic [ 11.893695] usbserial: USB Serial support registered for generic [ 11.916095] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 11.933625] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 11.935184] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 12.020359] mousedev: PS/2 mouse device common for all mice [ 12.029413] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 12.033460] rtc_cmos 00:05: RTC can wake from S4 [ 12.153239] rtc_cmos 00:05: registered as rtc0 [ 12.196769] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 12.199416] intel_pstate: CPU model not supported [ 12.203214] hid: raw HID events driver (C) Jiri Kosina [ 12.205730] usbcore: registered new interface driver usbhid [ 12.208030] usbhid: USB HID core driver [ 12.209861] drop_monitor: Initializing network drop monitor service [ 12.212207] Initializing XFRM netlink socket [ 12.214183] NET: Registered protocol family 10 [ 12.228474] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 12.232443] Segment Routing with IPv6 [ 12.233974] NET: Registered protocol family 17 [ 12.242509] mpls_gso: MPLS GSO support [ 12.278094] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 12.300077] RAS: Correctable Errors collector initialized. [ 12.312495] AVX version of gcm_enc/dec engaged. [ 12.319478] AES CTR mode by8 optimization enabled [ 12.929385] sched_clock: Marking stable (12929335906, 0)->(15268440828, -2339104922) [ 12.934876] registered taskstats version 1 [ 12.937402] Loading compiled-in X.509 certificates [ 12.940682] zswap: loaded using pool lzo/zbud [ 12.982826] Key type big_key registered [ 13.075314] Key type encrypted registered [ 13.078459] ima: No TPM chip found, activating TPM-bypass! [ 13.082833] ima: Allocated hash algorithm: sha1 [ 13.084905] ima: No architecture policies found [ 13.088803] evm: Initialising EVM extended attributes: [ 13.092017] evm: security.selinux [ 13.094624] evm: security.ima [ 13.096318] evm: security.capability [ 13.097452] evm: HMAC attrs: 0x1 [ 13.104246] rtc_cmos 00:05: setting system clock to 2026-05-08 22:07:48 UTC (1778278068) [ 13.122834] debug: unmapping init [mem 0xffffffff9ae03000-0xffffffff9affffff] [ 13.137803] debug: unmapping init [mem 0xffffffff99b82000-0xffffffff99e58fff] [ 13.175227] Write protecting the kernel read-only data: 28672k [ 13.211331] debug: unmapping init [mem 0xffffffff98203000-0xffffffff983fffff] [ 13.234422] debug: unmapping init [mem 0xffffffff98b14000-0xffffffff98bfffff] [ 13.548770] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.561847] systemd[1]: Detected virtualization kvm. [ 13.563989] systemd[1]: Detected architecture x86-64. [ 13.565776] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 13.597699] systemd[1]: No hostname configured. [ 13.599943] systemd[1]: Set hostname to . [ 13.602840] random: systemd: uninitialized urandom read (16 bytes read) [ 13.612709] systemd[1]: Initializing machine ID from random generator. [ 13.741451] random: ln: uninitialized urandom read (6 bytes read) [ 13.895676] random: systemd: uninitialized urandom read (16 bytes read) [ 13.899146] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 13.906655] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 13.914385] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 17.397844] device-mapper: uevent: version 1.0.3 [ 17.402257] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 23.766233] virtio_net virtio0 ens2: renamed from eth0 [ 24.785176] random: fast init done [ 27.681428] scsi host0: ata_piix [ 27.711136] scsi host1: ata_piix [ 27.712605] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 27.715079] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 33.487993] random: crng init done [ 33.575230] random: 7 urandom warning(s) missed due to ratelimiting [ 38.367892] dracut-initqueue[591]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 41.451052] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 47.297833] printk: systemd: 26 output lines suppressed due to ratelimiting [ 48.903910] SELinux: Disabled at runtime. [ 49.250345] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 49.287679] systemd[1]: Detected virtualization kvm. [ 49.308448] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 52.955202] systemd[1]: initrd-switch-root.service: Succeeded. [ 52.993291] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 53.058111] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 53.092990] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 53.103964] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 53.152421] systemd[1]: Starting Journal Service... Starting Journal Service... [ 53.253170] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. Starting Apply Kernel Variables... [ OK ] Created slice User and Session Slice. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target RPC Port Mapper. [ OK ] Reached target rpc_pipefs.target. Mounting Huge Pages File System... Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Stopped target Switch Root. [ OK ] Listening on udev Control Socket. [ 54.275548] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 54.325564] systemd[1]: Starting udev Coldplug all Devices... Starting udev Coldplug all Devices... [ 54.361715] systemd[1]: Created slice system-serial\x2dgetty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ 54.398964] systemd[1]: Stopped target Initrd File Systems. [ OK ] Stopped target Initrd File Systems. [ 54.451493] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ 54.504697] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 54.585204] systemd[1]: Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 54.651218] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ 54.749414] systemd[1]: Reached target Paths. [ OK ] Reached target Paths. [ 54.790490] systemd[1]: Reached target Local Encrypted Volumes. [ OK ] Reached target Local Encrypted Volumes. Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd Root File System. [ 55.150422] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 56.572487] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 59.449102] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 59.451497] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 60.921787] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 61.178927] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…only root support (10s / no limit) [** ] A start job is running for Configur…only root support (11s / no limit) [*** ] A start job is running for Configur…only root support (11s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ *** ] A start job is running for Configur…only root support (12s / no limit) [ ***] A start job is running for Configur…only root support (13s / no limit) [ **] A start job is running for Configur…only root support (13s / no limit) [ *] A start job is running for Configur…only root support (14s / no limit) [ **] A start job is running for Configur…only root support (14s / no limit) [ ***] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (15s / no limit) [ *** ] A start job is running for Configur…only root support (16s / no limit) [*** ] A start job is running for Configur…only root support (16s / no limit) [** ] A start job is running for Configur…only root support (17s / no limit) [* ] A start job is running for Configur…only root support (17s / no limit) [** ] A start job is running for Configur…only root support (18s / no limit) [*** ] A start job is running for Configur…only root support (18s / no limit)[ 71.865990] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (19s / no limit) [ *** ] A start job is running for Configur…only root support (19s / no limit) [ ***] A start job is running for Configur…only root support (20s / no limit) [ **] A start job is running for Configur…only root support (20s / no limit) [ *] A start job is running for Configur…only root support (21s / no limit)[ 73.935477] NFS: Registering the id_resolver key type [ 73.937537] Key type id_resolver registered [ 73.938974] Key type id_legacy registered [ **] A start job is running for Configur…only root support (21s / no limit) [ ***] A start job is running for Configur…only root support (22s / no limit) [ *** ] A start job is running for Configur…only root support (22s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ *** ] A start job is running for Rebuild …amic Linker Cache (29s / no limit) [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started OpenSSH server daemon. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg116-server login: [ 148.898936] libcfs: loading out-of-tree module taints kernel. [ 148.931667] Key type ._llcrypt registered [ 148.935128] Key type .llcrypt registered [ 149.033250] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 166.030170] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 167.406411] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 167.423029] alg: No test for adler32 (adler32-zlib) [ 168.771644] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 169.477053] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 171.175211] Key type lgssc registered [ 172.611451] Lustre: Echo OBD driver; http://www.lustre.org/ [ 188.506900] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 227.840996] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 240.364761] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 240.418390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 241.877391] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 241.940455] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 242.037840] Lustre: lustre-MDT0000: new disk, initializing [ 242.109209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 242.121223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 246.333564] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 259.284424] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 259.438613] Lustre: 6512:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 259.512501] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 259.526653] Lustre: Skipped 1 previous similar message [ 259.641567] Lustre: lustre-MDT0001: new disk, initializing [ 259.737944] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 259.763094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 259.774124] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 263.881224] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 268.391494] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 276.830993] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 277.040594] Lustre: lustre-OST0000: new disk, initializing [ 277.043405] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 277.091901] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 280.052940] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 280.067222] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 280.167459] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 283.366077] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 297.733764] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 297.946240] Lustre: lustre-OST0001: new disk, initializing [ 297.955264] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 298.070501] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 304.959591] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 305.241666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 305.263649] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 305.352773] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 317.073918] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 328.880883] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 341.044141] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing check_logdir /tmp/testlogs/ [ 344.953895] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing yml_node [ 348.623186] Lustre: DEBUG MARKER: Client: 2.17.52.6 [ 351.085528] Lustre: DEBUG MARKER: MDS: 2.17.52.6 [ 353.263784] Lustre: DEBUG MARKER: OSS: 2.17.52.6 [ 354.621787] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri May 8 18:13:29 EDT 2026 [ 370.160209] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 371.318305] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 387.552785] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 387.553292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 387.575854] Lustre: Skipped 1 previous similar message [ 387.585298] Lustre: Skipped 3 previous similar messages [ 392.682045] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 392.687668] Lustre: Skipped 3 previous similar messages [ 397.793101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 397.802712] Lustre: Skipped 3 previous similar messages [ 400.351193] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 400.592613] Lustre: server umount lustre-MDT0000 complete [ 403.915170] LustreError: 6505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278459 with bad export cookie 9695376576263990713 [ 403.924973] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 403.926161] LustreError: 6505:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 408.034618] LustreError: 6519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 408.036090] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 408.048289] LustreError: 6519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 408.068031] Lustre: Skipped 2 previous similar messages [ 408.081886] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 408.085489] Lustre: Skipped 1 previous similar message [ 413.152540] LustreError: 6521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 413.153314] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 413.184278] Lustre: Skipped 1 previous similar message [ 414.179230] LustreError: 6519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 414.206061] LustreError: 6519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 418.271134] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 418.538701] Lustre: server umount lustre-MDT0001 complete [ 422.072327] Lustre: server umount lustre-OST0000 complete [ 425.918659] Lustre: server umount lustre-OST0001 complete [ 431.695856] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 440.444970] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 473.081694] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 473.375563] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 473.425870] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 473.479365] Lustre: lustre-MDT0000: new disk, initializing [ 473.540262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 473.553065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 477.547323] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 487.037055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 487.160227] Lustre: 14987:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 487.209905] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 487.214352] Lustre: Skipped 1 previous similar message [ 487.297852] Lustre: lustre-MDT0001: new disk, initializing [ 487.412582] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 487.452326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 487.475697] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 491.601509] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 499.082051] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 501.818850] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 507.752666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 508.042143] Lustre: lustre-OST0000: new disk, initializing [ 508.045764] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 508.127368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 509.186721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 509.195424] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 509.271306] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 513.438514] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 519.414596] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 523.228049] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 523.392357] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 526.545680] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 526.756338] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 529.889212] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 529.895289] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 529.902820] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 529.915230] Lustre: Skipped 2 previous similar messages [ 529.924069] Lustre: Skipped 3 previous similar messages [ 534.359980] Lustre: server umount lustre-OST0000 complete [ 538.594920] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 538.598277] LustreError: Skipped 1 previous similar message [ 538.600590] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 542.891791] Lustre: server umount lustre-MDT0000 complete [ 545.255152] LustreError: 15000:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 545.274820] LustreError: 15000:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 545.841609] LustreError: 14980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278601 with bad export cookie 9695376576263992246 [ 545.852971] LustreError: 14980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 545.853797] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 546.073190] Lustre: server umount lustre-MDT0001 complete [ 549.952523] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:16:44 (1778278604) [ 555.301646] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 555.557465] LustreError: 18847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 555.623853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 559.009307] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 564.706363] LustreError: 18847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 564.715377] LustreError: 18847:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 565.232968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 568.446380] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 572.036807] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 574.612083] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 578.584851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 582.625976] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 586.131527] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 592.234296] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 599.191038] Lustre: server umount lustre-MDT0000 complete [ 600.547148] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 600.550424] LustreError: 18844:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.561393] Lustre: Skipped 3 previous similar messages [ 600.579522] LustreError: 18844:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 602.445382] LustreError: 18829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278657 with bad export cookie 9695376576263993128 [ 602.448187] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 602.457713] LustreError: 18829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 602.936134] Lustre: server umount lustre-MDT0001 complete [ 616.347199] Lustre: server umount lustre-OST0000 complete [ 631.920918] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 639.245460] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 639.577093] LustreError: 22872:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 639.610226] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 639.613017] Lustre: Skipped 2 previous similar messages [ 642.668634] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 647.987147] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 650.274576] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 651.887300] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 654.942497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 657.334218] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 660.747810] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 660.812916] Lustre: lustre-OST0001: new disk, initializing [ 660.814935] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 662.670597] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 662.675494] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 662.718107] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 663.075954] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 668.208475] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 669.641196] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 682.534870] Lustre: server umount lustre-MDT0000 complete [ 684.000991] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 684.007165] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 684.649317] LustreError: 22854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278740 with bad export cookie 9695376576263993975 [ 684.650231] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 684.655918] LustreError: 22854:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 686.858768] Lustre: server umount lustre-MDT0001 complete [ 698.902581] Lustre: server umount lustre-OST0000 complete [ 711.207962] Lustre: server umount lustre-OST0001 complete [ 718.938804] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 723.965620] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 724.177917] LustreError: 28125:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 724.191313] LustreError: 28125:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 725.907782] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 729.691334] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 731.548368] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 732.737367] Lustre: 29238:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 735.151501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 737.411826] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 740.430951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 742.471935] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 751.120039] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 752.357406] Lustre: 31090:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 759.265716] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 759.268065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 759.270199] Lustre: Skipped 6 previous similar messages [ 759.272337] Lustre: Skipped 5 previous similar messages [ 765.018320] Lustre: server umount lustre-MDT0000 complete [ 766.248098] LustreError: 28106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278821 with bad export cookie 9695376576263995270 [ 766.251335] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 766.251970] LustreError: 28106:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 766.365649] Lustre: server umount lustre-MDT0001 complete [ 777.729263] Lustre: server umount lustre-OST0000 complete [ 789.213538] Lustre: server umount lustre-OST0001 complete [ 792.011577] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:20:47 (1778278847) [ 801.113581] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 804.470423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 804.612867] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 804.616259] Lustre: Skipped 7 previous similar messages [ 805.838183] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 808.548178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 809.894334] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 810.889893] Lustre: 34983:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 812.962966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 814.928873] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 817.726764] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 819.553509] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 822.617945] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 823.706910] Lustre: 36831:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 834.528548] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 834.529079] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 834.533736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 834.535963] Lustre: Skipped 2 previous similar messages [ 834.537568] Lustre: Skipped 6 previous similar messages [ 835.990644] Lustre: server umount lustre-MDT0000 complete [ 837.110425] LustreError: 36833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278892 with bad export cookie 9695376576263996376 [ 837.112408] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 837.113624] LustreError: 36833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 837.219215] Lustre: server umount lustre-MDT0001 complete [ 847.875713] Lustre: server umount lustre-OST0000 complete [ 859.411958] Lustre: server umount lustre-OST0001 complete [ 862.050958] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:21:57 (1778278917) [ 866.786765] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 869.957179] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 870.089588] LustreError: 38954:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 870.096994] LustreError: 38954:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 871.342846] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 874.025015] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 875.363449] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 876.294470] Lustre: 40070:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 878.293512] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 880.150507] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 882.770029] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 884.613052] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 887.594594] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 893.985512] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 893.987009] Lustre: Skipped 1 previous similar message [ 894.944678] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 899.553101] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 899.553806] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 899.554796] Lustre: Skipped 4 previous similar messages [ 899.558128] Lustre: Skipped 3 previous similar messages [ 901.137651] Lustre: server umount lustre-MDT0000 complete [ 902.252212] LustreError: 38934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778278957 with bad export cookie 9695376576263997482 [ 902.255465] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 902.256004] LustreError: 38934:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 908.393570] Lustre: server umount lustre-MDT0001 complete [ 911.018703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 911.775631] LustreError: 3663:0:(client.c:1390:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff99177f658380 x1864659867351168/t0(0) o250->MGC192.168.201.116@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 913.148672] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 915.825769] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 917.242115] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 918.974345] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 932.385625] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 936.969069] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 17 sec [ 937.506303] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 937.508625] Lustre: Skipped 1 previous similar message [ 938.043399] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 938.571315] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 941.536290] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 945.945651] Lustre: server umount lustre-MDT0000 complete [ 947.087111] LustreError: 40073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279002 with bad export cookie 9695376576263998623 [ 947.089225] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 947.090551] LustreError: 40073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 947.192615] Lustre: server umount lustre-MDT0001 complete [ 958.592060] Lustre: server umount lustre-OST0000 complete [ 970.011260] Lustre: server umount lustre-OST0001 complete [ 972.631186] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:23:47 (1778279027) [ 977.403403] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 980.765683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 982.071284] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 984.917027] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 986.322541] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 987.304220] Lustre: 47422:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 987.308071] Lustre: 47422:0:(mgs_llog.c:1348:mgs_modify_param()) Skipped 1 previous similar message [ 989.472520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 991.301520] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 994.006164] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 996.133124] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 999.150576] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1005.664026] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1010.661056] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1015.963121] Lustre: server umount lustre-MDT0000 complete [ 1017.181174] LustreError: 46284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279072 with bad export cookie 9695376576263999442 [ 1017.184142] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1017.184655] LustreError: 46284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1017.296926] Lustre: server umount lustre-MDT0001 complete [ 1028.097681] Lustre: server umount lustre-OST0000 complete [ 1039.514309] Lustre: server umount lustre-OST0001 complete [ 1042.644675] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:24:57 (1778279097) [ 1043.412779] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1043.941647] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:24:59 (1778279099) [ 1047.291823] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1049.795609] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1056.752410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1056.849701] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1056.882926] Lustre: lustre-MDT0000: new disk, initializing [ 1056.909595] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1058.102183] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1060.986803] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1063.128427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1063.239996] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1063.242508] Lustre: Skipped 14 previous similar messages [ 1065.009118] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1065.086805] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1065.095771] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1067.956091] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1068.575462] Lustre: DEBUG MARKER: create test files [ 1080.800486] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1080.803696] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1080.807008] Lustre: Skipped 11 previous similar messages [ 1080.809059] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1080.810970] Lustre: Skipped 21 previous similar messages [ 1083.707605] Lustre: server umount lustre-OST0000 complete [ 1084.952214] Lustre: server umount lustre-MDT0000 complete [ 1088.213623] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 1089.176935] Key type lgssc unregistered [ 1089.299568] LNet: 54795:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1089.302052] LNetError: 54795:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1089.312267] LNet: Removed LNI 192.168.201.116@tcp [ 1089.623110] Key type .llcrypt unregistered [ 1089.624277] Key type ._llcrypt unregistered [ 1099.413109] Key type ._llcrypt registered [ 1099.414048] Key type .llcrypt registered [ 1099.450331] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1099.756946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1099.817344] alg: No test for adler32 (adler32-zlib) [ 1100.669836] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 1100.757393] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 1102.335139] Key type lgssc registered [ 1102.660887] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1105.130181] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1105.137877] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1106.303924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1107.590687] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1109.337085] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1111.465429] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1111.582827] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1113.387371] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1115.084306] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1115.683313] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1115.704467] Lustre: 57372:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1115.707424] Lustre: 57372:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1115.709759] Lustre: 57372:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1115.711889] Lustre: 57372:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/24/1, punch: 0/0/0, quota 1/3/0 [ 1115.714940] Lustre: 57372:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1115.716859] Lustre: 57372:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1123.239818] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1124.582677] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1130.977874] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1130.978276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1130.985465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1132.563140] Lustre: server umount lustre-OST0000 complete [ 1133.817876] Lustre: server umount lustre-MDT0000 complete [ 1137.036073] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 1137.966238] Key type lgssc unregistered [ 1138.087320] LNet: 58851:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1138.089463] LNetError: 58851:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1138.096263] LNet: Removed LNI 192.168.201.116@tcp [ 1138.361901] Key type .llcrypt unregistered [ 1138.363057] Key type ._llcrypt unregistered [ 1148.877843] Key type ._llcrypt registered [ 1148.878910] Key type .llcrypt registered [ 1148.917330] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1149.289305] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1149.329644] alg: No test for adler32 (adler32-zlib) [ 1150.179836] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 1150.249276] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 1151.831160] Key type lgssc registered [ 1152.153904] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1154.492523] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1154.500890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1155.637426] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1156.805219] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1158.380537] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1160.406455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1160.500609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1162.239133] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1163.896973] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1166.627823] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1166.645739] Lustre: 61525:0:(osd_internal.h:1446:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1166.649442] Lustre: 61525:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1166.653689] Lustre: 61525:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1166.656700] Lustre: 61525:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/24/1, punch: 0/0/0, quota 1/3/0 [ 1166.658795] Lustre: 61525:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1166.660515] Lustre: 61525:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1171.980835] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1177.056842] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1177.056945] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1177.064601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1180.241809] Lustre: server umount lustre-OST0000 complete [ 1181.612845] Lustre: server umount lustre-MDT0000 complete [ 1185.097262] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 1186.048822] Key type lgssc unregistered [ 1186.171388] LNet: 62953:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1186.173544] LNetError: 62953:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1186.183260] LNet: Removed LNI 192.168.201.116@tcp [ 1186.457108] Key type .llcrypt unregistered [ 1186.458360] Key type ._llcrypt unregistered [ 1192.041361] Key type ._llcrypt registered [ 1192.042539] Key type .llcrypt registered [ 1192.076931] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1197.807468] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1198.091844] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1198.121346] alg: No test for adler32 (adler32-zlib) [ 1198.971310] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 1199.048875] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 1200.623113] Key type lgssc registered [ 1200.929493] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1211.864333] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1211.869108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1212.971119] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1212.983262] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1213.021189] Lustre: lustre-MDT0000: new disk, initializing [ 1213.043709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1213.049762] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1214.300041] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1218.252646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1218.291887] Lustre: 66045:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1218.315951] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1218.317495] Lustre: Skipped 1 previous similar message [ 1218.355275] Lustre: lustre-MDT0001: new disk, initializing [ 1218.386777] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1218.400205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1218.404459] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1219.671123] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1222.677525] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1223.793684] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1225.936672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1226.036253] Lustre: lustre-OST0000: new disk, initializing [ 1226.038662] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1226.072085] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1227.764426] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1227.768698] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1227.800562] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1227.950662] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1230.856132] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1232.105581] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1232.167434] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1233.262207] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1233.325988] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1237.985652] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1237.985835] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1237.989225] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1237.992814] Lustre: Skipped 1 previous similar message [ 1240.017372] Lustre: server umount lustre-OST0000 complete [ 1243.106467] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1243.107184] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1243.110263] Lustre: Skipped 2 previous similar messages [ 1247.379321] Lustre: server umount lustre-MDT0000 complete [ 1248.508255] LustreError: 66036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279303 with bad export cookie 14886838969185875556 [ 1248.511035] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1248.512626] LustreError: 66036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1248.619052] Lustre: server umount lustre-MDT0001 complete [ 1251.214928] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:28:26 (1778279306) [ 1254.543511] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1266.528694] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1269.502087] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1280.526727] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1280.610168] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1280.620768] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1280.653565] Lustre: lustre-MDT0000: new disk, initializing [ 1280.674533] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1280.681430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1281.834507] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1285.712273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1285.745745] Lustre: 73597:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1285.767210] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1285.768690] Lustre: Skipped 1 previous similar message [ 1285.799941] Lustre: lustre-MDT0001: new disk, initializing [ 1285.825899] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1285.835706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1285.838524] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1286.970727] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1289.770101] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1290.881526] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1292.928567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1294.795858] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1296.632820] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1296.638057] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1296.675376] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1297.826744] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1299.175778] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1299.249378] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1300.531117] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1300.601412] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1301.987683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1301.987718] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1301.988891] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1301.988896] Lustre: Skipped 1 previous similar message [ 1301.991109] Lustre: Skipped 1 previous similar message [ 1301.997447] LustreError: Skipped 2 previous similar messages [ 1303.313315] Lustre: server umount lustre-OST0000 complete [ 1306.591812] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1306.594117] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1306.598054] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1306.601039] Lustre: Skipped 1 previous similar message [ 1310.679184] Lustre: server umount lustre-MDT0000 complete [ 1311.933300] LustreError: 74372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279367 with bad export cookie 14886838969185876424 [ 1311.938083] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1311.938755] LustreError: 74372:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1312.063757] Lustre: server umount lustre-MDT0001 complete [ 1314.819606] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:29:30 (1778279370) [ 1317.039925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1317.184149] LustreError: 77595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1317.191084] LustreError: 77595:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1317.209531] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1317.212662] Lustre: Skipped 1 previous similar message [ 1318.567440] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1321.409488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1322.834675] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1324.550196] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1325.771783] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1327.976028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1329.928529] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1331.683277] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1332.939162] Lustre: 79927:0:(genops.c:1793:obd_export_evict_by_uuid()) MGS: evicting 51d9bf1e-2e3d-44e6-95c2-d13cef2fc9db at adminstrative request [ 1336.290872] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1336.294680] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1346.530237] Lustre: MGS: Client 51d9bf1e-2e3d-44e6-95c2-d13cef2fc9db (at 0@lo) reconnecting [ 1346.534646] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xce98ad2b2ac43d2c to 0xce98ad2b2ac43fb7 [ 1346.539197] Lustre: MGC192.168.201.116@tcp: Connection restored to 0@lo (at 0@lo) [ 1365.392729] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1365.462195] Lustre: lustre-OST0001: new disk, initializing [ 1365.464395] Lustre: Skipped 1 previous similar message [ 1365.466974] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1365.469755] Lustre: Skipped 1 previous similar message [ 1365.499026] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1365.501398] Lustre: Skipped 2 previous similar messages [ 1367.386649] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1370.510094] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1371.493515] Lustre: server umount lustre-OST0001 complete [ 1373.151768] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1373.154105] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1373.157691] Lustre: Skipped 1 previous similar message [ 1373.159331] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1373.161134] Lustre: Skipped 2 previous similar messages [ 1375.212602] LustreError: 79190:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1380.326446] LustreError: 79188:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1380.331995] LustreError: 79188:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1385.442596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1385.443478] LustreError: 81258:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1385.445717] Lustre: Skipped 5 previous similar messages [ 1385.450778] LustreError: 81258:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1386.975085] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1387.031370] Lustre: server umount lustre-OST0000 complete [ 1390.562451] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1390.565764] Lustre: Skipped 1 previous similar message [ 1394.458033] Lustre: server umount lustre-MDT0000 complete [ 1395.664380] LustreError: 77577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279451 with bad export cookie 14886838969185877943 [ 1395.667225] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1395.669886] LustreError: 77577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1395.789665] Lustre: server umount lustre-MDT0001 complete [ 1398.374746] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:30:53 (1778279453) [ 1398.903962] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1399.505548] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:30:54 (1778279454) [ 1400.063617] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1400.713404] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:30:55 (1778279455) [ 1401.266923] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1401.827813] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:30:57 (1778279457) [ 1402.319024] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1402.860687] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:30:58 (1778279458) [ 1403.377928] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1405.348297] loop: module loaded [ 1405.353317] loop0: detected capacity change from 0 to 4294967296 [ 1405.365602] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1405.867718] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1406.819037] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1408.047916] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1416.793492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1416.808586] LustreError: 84308:0:(obd_config.c:843:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1416.810514] LustreError: 84308:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1416.812564] LustreError: 84308:0:(tgt_mount.c:2513:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1416.814791] LustreError: 84308:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1422.923348] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1426.319868] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1438.757722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1438.867945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1438.879391] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1438.923065] Lustre: lustre-MDT0000: new disk, initializing [ 1438.946691] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1438.952841] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1440.345722] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1444.559633] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1444.606961] Lustre: 87131:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1444.727106] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1446.095747] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1449.158348] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1450.389844] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1452.656455] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1454.655428] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1457.695809] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1459.019145] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1459.701812] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1459.732762] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1460.106829] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 1 sec [ 1461.374712] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1461.444395] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1464.804616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1464.805905] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1464.807598] LustreError: Skipped 1 previous similar message [ 1464.811570] Lustre: Skipped 1 previous similar message [ 1464.814421] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1464.817671] Lustre: Skipped 3 previous similar messages [ 1468.175117] Lustre: server umount lustre-OST0000 complete [ 1475.549337] Lustre: server umount lustre-MDT0000 complete [ 1476.781766] LustreError: 87911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279532 with bad export cookie 14886838969185878524 [ 1476.784821] LustreError: 87911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1476.784959] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1476.909773] Lustre: server umount lustre-MDT0001 complete [ 1479.836405] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:32:15 (1778279535) [ 1482.076432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1482.243979] LustreError: 91138:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1483.544374] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1486.361658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1487.792792] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1489.555515] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1490.762525] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1492.990119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1494.946786] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1496.713150] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1498.288462] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1507.667797] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 9 sec [ 1509.562793] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1509.671061] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1512.740552] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1512.818530] Lustre: lustre-OST0001: new disk, initializing [ 1512.820806] Lustre: Skipped 2 previous similar messages [ 1512.823406] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1512.825883] Lustre: Skipped 3 previous similar messages [ 1512.865101] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1512.867536] Lustre: Skipped 5 previous similar messages [ 1515.356545] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1518.098644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1518.107175] Lustre: Skipped 2 previous similar messages [ 1518.111521] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1518.154963] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1519.348754] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 1521.471776] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1521.559727] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1523.416235] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1523.508989] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1533.845481] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1538.824137] Lustre: Failing over lustre-MDT0000 [ 1539.052485] Lustre: server umount lustre-MDT0000 complete [ 1540.303688] LustreError: 92498:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1540.307965] LustreError: 92498:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 1543.135801] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1543.137610] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1543.140979] Lustre: Skipped 2 previous similar messages [ 1553.821824] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1553.822972] LDISKFS-fs (dm-0): recovery complete [ 1553.825422] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1553.865160] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1553.889727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (not set up) [ 1553.891641] Lustre: Skipped 5 previous similar messages [ 1553.998629] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1555.286489] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1555.656750] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1555.660316] Lustre: 91134:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1559.012192] Lustre: 91138:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1559.015677] Lustre: 91138:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 1 previous similar message [ 1559.020306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1559.039334] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1559.351124] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1559.665525] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1559.669690] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1559.983076] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1559.985801] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 1 previous similar message [ 1560.305181] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1560.308860] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 8 previous similar messages [ 1560.937172] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1560.939652] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1561.255132] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1561.258315] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 3 previous similar messages [ 1562.511794] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1562.515071] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 6 previous similar messages [ 1563.143959] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1563.146495] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1563.463083] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1563.465604] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 6 previous similar messages [ 1566.593478] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1566.597245] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 12 previous similar messages [ 1567.225151] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1567.227941] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1567.543136] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1567.545390] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 12 previous similar messages [ 1574.728951] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1574.732014] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 25 previous similar messages [ 1575.361197] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1575.363964] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1575.679144] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1575.681622] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 25 previous similar messages [ 1590.985111] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1590.990132] Lustre: 96877:0:(ldlm_lib.c:2068:extend_recovery_timer()) Skipped 51 previous similar messages [ 1591.617341] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1591.622545] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1591.935138] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1591.938818] LustreError: 96877:0:(ldlm_lib.c:2681:replay_request_or_update()) Skipped 51 previous similar messages [ 1615.031137] Lustre: 96877:0:(ldlm_lib.c:2377:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1615.035447] Lustre: 96877:0:(ldlm_lib.c:2387:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1615.038599] Lustre: 96877:0:(genops.c:1620:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1876dc01-778a-4149-8fb6-eb7a57f7dd4b@192.168.201.16@tcp [ 1615.041790] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1615.043487] Lustre: 96877:0:(ldlm_lib.c:1897:abort_req_replay_queue()) @@@ aborted: req@0000000076352645 x1864660941866624/t0(8589934776) o35->1876dc01-778a-4149-8fb6-eb7a57f7dd4b@192.168.201.16@tcp:246/0 lens 392/0 e 0 to 0 dl 1778279681 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1615.051086] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1615.053193] Lustre: lustre-MDT0000: Denying connection for new client 1876dc01-778a-4149-8fb6-eb7a57f7dd4b (at 192.168.201.16@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1615.056863] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1615.068420] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1615.068437] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1615.071018] Lustre: Skipped 2 previous similar messages [ 1615.084938] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 1615.084973] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 1620.450595] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1620.451363] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1620.457967] Lustre: Skipped 3 previous similar messages [ 1620.460518] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1620.462734] Lustre: Skipped 1 previous similar message [ 1626.301476] Lustre: server umount lustre-OST0000 complete [ 1630.695131] LustreError: 93237:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1630.700614] LustreError: 93237:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 1633.684403] Lustre: server umount lustre-OST0001 complete [ 1641.056718] Lustre: server umount lustre-MDT0000 complete [ 1642.238315] LustreError: 91120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279697 with bad export cookie 14886838969185929708 [ 1642.240855] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1642.241331] LustreError: 91120:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1642.355584] Lustre: server umount lustre-MDT0001 complete [ 1645.022236] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:35:00 (1778279700) [ 1647.382770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1647.385893] Lustre: *** cfs_fail_loc=197, val=0*** [ 1662.815564] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1667.935452] LustreError: 98835:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.116@tcp: failed processing log, type 4: rc = -110 [ 1673.063294] Lustre: *** cfs_fail_loc=197, val=0*** [ 1673.064815] Lustre: Skipped 65 previous similar messages [ 1693.535396] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1693.537562] Lustre: Skipped 1 previous similar message [ 1695.426519] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1697.213718] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1697.929567] Lustre: server umount lustre-OST0000 complete [ 1700.655224] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:35:55 (1778279755) [ 1714.389530] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1714.559584] LustreError: 100340:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1715.916826] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1718.946903] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1720.943826] LustreError: 100371:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1720.949834] LustreError: 100371:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1720.952921] LustreError: 100371:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1721.096428] Lustre: server umount lustre-MDT0000 complete [ 1731.642512] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1734.888215] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1746.955519] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1747.050781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1747.061641] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1747.101813] Lustre: lustre-MDT0000: new disk, initializing [ 1747.163334] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1748.549315] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1752.679826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1752.718905] Lustre: 104387:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1752.818441] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1754.180115] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1757.329915] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1758.483578] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1760.732744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1762.850971] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1763.357556] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1765.993447] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1767.358756] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1767.432488] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1768.807435] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1768.881570] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1773.540449] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1773.540652] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1773.542985] LustreError: Skipped 4 previous similar messages [ 1773.546496] Lustre: Skipped 5 previous similar messages [ 1773.547237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1773.553377] Lustre: Skipped 9 previous similar messages [ 1775.632407] Lustre: server umount lustre-OST0000 complete [ 1783.006490] Lustre: server umount lustre-MDT0000 complete [ 1783.776175] LustreError: 105171:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1783.780867] LustreError: 105171:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 1784.282439] LustreError: 105165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279839 with bad export cookie 14886838969185940355 [ 1784.286557] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1784.287571] LustreError: 105165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1784.396052] Lustre: server umount lustre-MDT0001 complete [ 1787.195864] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:37:22 (1778279842) [ 1790.150697] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 1791.208292] Key type lgssc unregistered [ 1791.350387] LNet: 108449:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1791.353756] LNetError: 108449:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1791.363424] LNet: Removed LNI 192.168.201.116@tcp [ 1791.700727] Key type .llcrypt unregistered [ 1791.701919] Key type ._llcrypt unregistered [ 1797.756564] Key type ._llcrypt registered [ 1797.757769] Key type .llcrypt registered [ 1797.798862] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1804.002194] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1804.448801] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1804.509892] alg: No test for adler32 (adler32-zlib) [ 1805.380125] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 1805.476956] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 1807.063141] Key type lgssc registered [ 1807.417362] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1820.766647] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1820.772674] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1821.867979] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1821.881214] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1821.917637] Lustre: lustre-MDT0000: new disk, initializing [ 1821.940898] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1821.947123] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1823.161150] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1827.241771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1827.355390] Lustre: lustre-OST0000: new disk, initializing [ 1827.357303] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1827.359123] Lustre: Skipped 1 previous similar message [ 1827.382356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.238477] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1832.957103] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1832.960274] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1832.967857] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1833.382501] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1833.456320] Lustre: lustre-OST0001: new disk, initializing [ 1833.459825] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1833.492465] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1835.485460] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1843.680399] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1843.680875] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1843.684283] Lustre: Skipped 1 previous similar message [ 1843.686857] Lustre: Skipped 1 previous similar message [ 1845.621174] Lustre: server umount lustre-MDT0000 complete [ 1847.835875] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1849.372285] LustreError: 111745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279904 with bad export cookie 8326803116493034500 [ 1849.374424] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1849.375929] LustreError: 111745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1849.421454] Lustre: server umount lustre-OST0000 complete [ 1850.703972] Lustre: server umount lustre-OST0001 complete [ 1853.622799] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 18:38:28 (1778279908) [ 1856.887460] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1864.543334] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:38:39 (1778279919) [ 1877.311335] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 1880.689158] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 1892.342792] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1892.442692] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1892.456818] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1892.504885] Lustre: lustre-MDT0000: new disk, initializing [ 1892.532250] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1892.538854] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1893.919562] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1898.171736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1898.220524] Lustre: 120056:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1898.247843] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1898.249655] Lustre: Skipped 1 previous similar message [ 1898.286467] Lustre: lustre-MDT0001: new disk, initializing [ 1898.322899] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1898.337859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1898.341065] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1899.679835] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1902.801943] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1904.079500] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1906.357616] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1906.487400] Lustre: lustre-OST0000: new disk, initializing [ 1906.489803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1906.525956] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1908.551921] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1909.181570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1909.185346] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1909.220683] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1911.656265] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1913.057134] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1913.136060] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1914.550392] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1914.634211] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1919.457603] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1919.458336] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1919.459701] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1919.461498] LustreError: Skipped 1 previous similar message [ 1919.464796] Lustre: Skipped 1 previous similar message [ 1921.424778] Lustre: server umount lustre-OST0000 complete [ 1924.064158] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1924.068216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1924.073960] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1924.075708] Lustre: Skipped 1 previous similar message [ 1928.855434] Lustre: server umount lustre-MDT0000 complete [ 1930.129212] LustreError: 120048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778279985 with bad export cookie 8326803116493035655 [ 1930.133720] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1930.243930] Lustre: server umount lustre-MDT0001 complete [ 1933.083926] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:39:48 (1778279988) [ 1935.409608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1935.573196] LustreError: 124057:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1935.579662] LustreError: 124057:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 1935.598092] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1936.884749] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1939.748950] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1941.346677] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1943.244105] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1944.447801] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1946.845798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1948.801069] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 1950.548766] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1963.198988] Lustre: *** cfs_fail_loc=159, val=0*** [ 1978.581248] Lustre: lustre-MDT0000: Client 20cebac9-66c5-48a2-b330-dfa6b33fb605 (at 192.168.201.16@tcp) reconnecting [ 1979.122644] Lustre: *** cfs_fail_loc=159, val=0*** [ 1979.124262] Lustre: Skipped 6 previous similar messages [ 1994.451357] Lustre: lustre-MDT0000: Client 20cebac9-66c5-48a2-b330-dfa6b33fb605 (at 192.168.201.16@tcp) reconnecting [ 2011.617317] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2011.617487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2011.620214] LustreError: Skipped 1 previous similar message [ 2011.620850] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2011.625239] Lustre: Skipped 2 previous similar messages [ 2011.629842] Lustre: Skipped 2 previous similar messages [ 2016.532912] Lustre: server umount lustre-OST0000 complete [ 2021.859678] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2021.860170] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2021.862791] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2021.867139] Lustre: Skipped 1 previous similar message [ 2021.870593] Lustre: Skipped 1 previous similar message [ 2024.021182] Lustre: server umount lustre-MDT0000 complete [ 2025.205057] LustreError: 124037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280080 with bad export cookie 8326803116493036523 [ 2025.208030] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2025.208093] LustreError: 124037:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 2025.333754] Lustre: server umount lustre-MDT0001 complete [ 2028.808730] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2029.756191] Key type lgssc unregistered [ 2029.898308] LNet: 127502:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2029.900353] LNetError: 127502:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2029.908330] LNet: Removed LNI 192.168.201.116@tcp [ 2030.180960] Key type .llcrypt unregistered [ 2030.182542] Key type ._llcrypt unregistered [ 2039.338707] Key type ._llcrypt registered [ 2039.339675] Key type .llcrypt registered [ 2039.379548] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2039.691167] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2039.720057] alg: No test for adler32 (adler32-zlib) [ 2040.572525] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2040.653726] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2042.231314] Key type lgssc registered [ 2042.612058] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2045.328198] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2045.336049] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2046.496990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2047.976231] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2051.256886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2051.381595] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2052.654353] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2054.409687] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2055.595950] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2058.063728] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2058.203486] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2060.008706] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2061.843926] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2071.534939] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2076.761054] Lustre: *** cfs_fail_loc=159, val=0*** [ 2093.272916] Lustre: lustre-MDT0000: Client 73f69720-659e-4a3c-b4fb-ca1f81f5be7c (at 192.168.201.16@tcp) reconnecting [ 2093.969736] Lustre: *** cfs_fail_loc=159, val=0*** [ 2110.164636] Lustre: lustre-MDT0001: Client 73f69720-659e-4a3c-b4fb-ca1f81f5be7c (at 192.168.201.16@tcp) reconnecting [ 2110.683698] Lustre: *** cfs_fail_loc=159, val=0*** [ 2110.685596] Lustre: Skipped 4 previous similar messages [ 2127.060849] Lustre: lustre-MDT0001: Client 73f69720-659e-4a3c-b4fb-ca1f81f5be7c (at 192.168.201.16@tcp) reconnecting [ 2128.313722] Lustre: 131593:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2129.125607] Lustre: *** cfs_fail_loc=159, val=0*** [ 2129.126788] Lustre: Skipped 3 previous similar messages [ 2144.473280] Lustre: lustre-MDT0000: Client 778b1fb3-7566-465f-8261-81d0d77519a5 (at 192.168.201.16@tcp) reconnecting [ 2145.020370] Lustre: *** cfs_fail_loc=159, val=0*** [ 2145.021652] Lustre: Skipped 14 previous similar messages [ 2160.342336] Lustre: lustre-MDT0000: Client 778b1fb3-7566-465f-8261-81d0d77519a5 (at 192.168.201.16@tcp) reconnecting [ 2160.831588] Lustre: 131883:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2163.683824] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2163.686972] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2163.689247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2163.696380] Lustre: Skipped 1 previous similar message [ 2168.138208] Lustre: server umount lustre-OST0000 complete [ 2169.311710] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2169.313838] LustreError: Skipped 1 previous similar message [ 2169.315039] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2169.318470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2169.320048] Lustre: Skipped 1 previous similar message [ 2173.920201] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2173.920589] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2175.447993] Lustre: server umount lustre-MDT0000 complete [ 2176.692202] LustreError: 128857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280232 with bad export cookie 9691451125726049442 [ 2176.695188] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2176.695327] LustreError: 128857:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2176.836088] Lustre: server umount lustre-MDT0001 complete [ 2180.474196] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2181.470490] Key type lgssc unregistered [ 2181.585339] LNet: 132916:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2181.587654] LNetError: 132916:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2181.596290] LNet: Removed LNI 192.168.201.116@tcp [ 2181.878965] Key type .llcrypt unregistered [ 2181.880386] Key type ._llcrypt unregistered [ 2190.710992] Key type ._llcrypt registered [ 2190.711899] Key type .llcrypt registered [ 2190.743493] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2191.144549] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2191.172527] alg: No test for adler32 (adler32-zlib) [ 2192.027713] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2192.117438] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2193.703147] Key type lgssc registered [ 2194.106496] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2196.752919] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2196.758844] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2197.935334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2199.238692] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2201.970866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2202.102157] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2203.357358] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2204.988111] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2206.228253] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2208.423561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2208.552338] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2210.389255] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2211.623159] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2212.034757] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2222.883967] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2230.026567] Lustre: 136560:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2230.628776] Lustre: 136608:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2231.059730] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2232.118156] Lustre: 136753:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2232.403277] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2232.405669] Lustre: Skipped 1 previous similar message [ 2232.687755] Lustre: 136851:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2237.408216] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2237.412499] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2237.420833] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2238.433135] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2238.433423] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2238.435758] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2239.691865] Lustre: server umount lustre-OST0000 complete [ 2243.039708] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2243.041601] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2243.044719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2243.046083] Lustre: Skipped 1 previous similar message [ 2248.672730] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2248.674336] Lustre: Skipped 2 previous similar messages [ 2253.792679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2253.795197] Lustre: Skipped 1 previous similar message [ 2255.327122] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2255.390431] Lustre: server umount lustre-MDT0000 complete [ 2256.581584] LustreError: 134314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280311 with bad export cookie 1497516689715248300 [ 2256.584591] LustreError: 134314:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2256.586251] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2256.709753] Lustre: server umount lustre-MDT0001 complete [ 2260.423423] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2261.469335] Key type lgssc unregistered [ 2261.594611] LNet: 137890:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2261.597304] LNetError: 137890:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2261.604527] LNet: Removed LNI 192.168.201.116@tcp [ 2261.919710] Key type .llcrypt unregistered [ 2261.920827] Key type ._llcrypt unregistered [ 2270.735601] Key type ._llcrypt registered [ 2270.736580] Key type .llcrypt registered [ 2270.773116] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2271.254183] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2271.286520] alg: No test for adler32 (adler32-zlib) [ 2272.140350] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2272.225988] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2273.807197] Key type lgssc registered [ 2274.185596] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2276.692951] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2276.698598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2277.869908] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2279.020509] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2281.776154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2281.906624] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2283.135356] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2284.986082] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2286.100445] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2288.392044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2288.527046] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2290.393574] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2291.626271] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2292.164230] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2302.889553] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2309.855336] Lustre: *** cfs_fail_loc=159, val=0*** [ 2326.244662] Lustre: lustre-MDT0000: Client 979c3e55-d090-4620-9916-e58f544ed0c2 (at 192.168.201.16@tcp) reconnecting [ 2327.519753] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2327.523029] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2327.526769] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2328.545625] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2328.547551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2328.547560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2333.518176] Lustre: server umount lustre-OST0000 complete [ 2338.272070] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2338.274220] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2338.277818] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2338.279334] Lustre: Skipped 1 previous similar message [ 2343.904713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2343.908150] Lustre: Skipped 2 previous similar messages [ 2349.023078] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2349.024597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2349.027782] Lustre: Skipped 1 previous similar message [ 2349.092394] Lustre: server umount lustre-MDT0000 complete [ 2350.377327] LustreError: 139989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280405 with bad export cookie 13203787608758547684 [ 2350.382552] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2350.385744] LustreError: 139989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2350.536823] Lustre: server umount lustre-MDT0001 complete [ 2354.000132] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2354.961905] Key type lgssc unregistered [ 2355.080455] LNet: 142597:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2355.082378] LNetError: 142597:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2355.089396] LNet: Removed LNI 192.168.201.116@tcp [ 2355.379879] Key type .llcrypt unregistered [ 2355.380935] Key type ._llcrypt unregistered [ 2364.069828] Key type ._llcrypt registered [ 2364.071784] Key type .llcrypt registered [ 2364.113227] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2364.465177] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2364.484570] alg: No test for adler32 (adler32-zlib) [ 2365.335875] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2365.416683] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2366.999118] Key type lgssc registered [ 2367.341884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2369.988526] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2369.995059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2371.154936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2372.420365] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2375.659396] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2375.788679] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2377.064594] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2378.735969] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2379.826593] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2382.003704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2382.146412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2383.933105] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2385.600798] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2386.277544] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2396.519103] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2414.089524] Lustre: lustre-OST0000: evicting 698f05bd-b384-4bff-8776-9090aa2bb6c8 (at 192.168.201.16@tcp) by administrative request [ 2414.091989] Lustre: Skipped 1 previous similar message [ 2440.674074] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2440.674966] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2440.677303] Lustre: Skipped 1 previous similar message [ 2444.174697] Lustre: server umount lustre-OST0000 complete [ 2445.792795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2445.793567] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2445.795882] Lustre: Skipped 1 previous similar message [ 2445.797498] Lustre: Skipped 1 previous similar message [ 2450.912435] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2450.915987] Lustre: Skipped 2 previous similar messages [ 2451.540077] Lustre: server umount lustre-MDT0000 complete [ 2452.732385] LustreError: 143950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280508 with bad export cookie 8233828365753891594 [ 2452.735677] LustreError: 143950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2452.736425] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2452.857263] Lustre: server umount lustre-MDT0001 complete [ 2456.329638] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2457.337806] Key type lgssc unregistered [ 2457.466477] LNet: 147702:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2457.469752] LNetError: 147702:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2457.480443] LNet: Removed LNI 192.168.201.116@tcp [ 2457.771116] Key type .llcrypt unregistered [ 2457.772226] Key type ._llcrypt unregistered [ 2480.630877] Key type ._llcrypt registered [ 2480.631925] Key type .llcrypt registered [ 2480.662949] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2481.034993] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2481.051776] alg: No test for adler32 (adler32-zlib) [ 2481.913786] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2481.998968] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2483.575148] Key type lgssc registered [ 2483.927372] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2486.499920] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2486.507587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2487.694780] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2488.947815] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2491.642688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2491.788215] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2493.153819] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2494.868277] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2495.978912] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2498.159851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2498.273657] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2500.124893] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2501.829097] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2502.372706] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2512.613213] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2523.106118] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2523.106470] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2523.108787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2523.112015] Lustre: Skipped 1 previous similar message [ 2523.115089] Lustre: Skipped 1 previous similar message [ 2526.353873] Lustre: server umount lustre-OST0000 complete [ 2527.711740] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2527.713863] LustreError: Skipped 1 previous similar message [ 2527.715058] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2527.719234] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2533.343922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2533.345568] Lustre: Skipped 3 previous similar messages [ 2533.650884] Lustre: server umount lustre-MDT0000 complete [ 2534.865038] LustreError: 150127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280590 with bad export cookie 11397448062392839467 [ 2534.867801] LustreError: 150127:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2534.867895] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2534.972384] Lustre: server umount lustre-MDT0001 complete [ 2538.521836] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2539.454415] Key type lgssc unregistered [ 2539.569331] LNet: 153354:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2539.571486] LNetError: 153354:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2539.577308] LNet: Removed LNI 192.168.201.116@tcp [ 2539.835655] Key type .llcrypt unregistered [ 2539.837109] Key type ._llcrypt unregistered [ 2548.671520] Key type ._llcrypt registered [ 2548.672452] Key type .llcrypt registered [ 2548.707207] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2549.040442] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2549.091712] alg: No test for adler32 (adler32-zlib) [ 2549.934350] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2550.006734] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2551.583178] Key type lgssc registered [ 2551.971376] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2557.323223] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:50:12 (1778280612) [ 2562.570956] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 2565.815359] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2577.412252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2577.416738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2578.506160] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2578.517050] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2578.559758] Lustre: lustre-MDT0000: new disk, initializing [ 2578.584069] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2578.590370] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2579.866127] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2583.981652] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2584.016308] Lustre: 157535:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2584.037109] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2584.038723] Lustre: Skipped 1 previous similar message [ 2584.073030] Lustre: lustre-MDT0001: new disk, initializing [ 2584.100946] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2584.110453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2584.113113] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2585.380393] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2588.369104] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2589.545571] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2591.716731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2591.810162] Lustre: lustre-OST0000: new disk, initializing [ 2591.812046] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2591.837424] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2593.593378] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2593.914177] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2593.916865] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2593.950676] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2596.702181] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2604.513900] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2604.514636] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2604.515876] LustreError: Skipped 1 previous similar message [ 2604.521427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2604.523447] Lustre: Skipped 1 previous similar message [ 2605.834988] Lustre: server umount lustre-OST0000 complete [ 2609.633977] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2609.634501] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2609.635526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2609.640717] Lustre: Skipped 2 previous similar messages [ 2614.752601] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2614.754241] Lustre: Skipped 2 previous similar messages [ 2619.872567] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2619.874301] Lustre: Skipped 1 previous similar message [ 2621.407086] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2621.468377] Lustre: server umount lustre-MDT0000 complete [ 2622.677127] LustreError: 157527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778280678 with bad export cookie 5969492900360810420 [ 2622.679769] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2622.680169] LustreError: 157527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2622.785697] Lustre: server umount lustre-MDT0001 complete [ 2625.904990] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 2626.856069] Key type lgssc unregistered [ 2626.974314] LNet: 161048:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2626.976607] LNetError: 161048:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2626.983321] LNet: Removed LNI 192.168.201.116@tcp [ 2627.261723] Key type .llcrypt unregistered [ 2627.262976] Key type ._llcrypt unregistered [ 2635.426783] Key type ._llcrypt registered [ 2635.427772] Key type .llcrypt registered [ 2635.461873] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 2635.781944] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2635.822681] alg: No test for adler32 (adler32-zlib) [ 2636.678504] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 2636.759636] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 2638.335135] Key type lgssc registered [ 2638.675924] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2641.435130] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2641.447328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2642.619530] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2643.864890] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2646.600964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2646.721064] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2647.958833] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2649.607150] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2650.758810] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2652.830793] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2652.932963] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2653.990470] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2654.601817] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 2656.198264] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2659.509251] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2659.516121] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2659.520635] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2659.523645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.528954] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2659.841365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.168110] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2660.170244] Lustre: Skipped 1 previous similar message [ 2660.174633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2660.177753] Lustre: Skipped 1 previous similar message [ 2660.179040] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2660.180730] Lustre: Skipped 1 previous similar message [ 2660.181922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.184815] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2660.186477] Lustre: Skipped 1 previous similar message [ 2660.469553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.749355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.026371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.310524] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2661.312444] Lustre: Skipped 3 previous similar messages [ 2661.316870] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2661.320554] Lustre: Skipped 3 previous similar messages [ 2661.321838] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2661.323750] Lustre: Skipped 3 previous similar messages [ 2661.325069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.329256] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2661.332318] Lustre: Skipped 3 previous similar messages [ 2661.610553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.962205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.298402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.639401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.937970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.234282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.509270] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2663.511379] Lustre: Skipped 6 previous similar messages [ 2663.517154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2663.520618] Lustre: Skipped 6 previous similar messages [ 2663.522033] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2663.523835] Lustre: Skipped 6 previous similar messages [ 2663.525226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.528921] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2663.530751] Lustre: Skipped 6 previous similar messages [ 2663.819816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.095291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.397935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.697458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.003791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.287469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.578946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.871641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.165773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.446158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.748247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.065605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.379632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.666331] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2667.669556] Lustre: Skipped 13 previous similar messages [ 2667.675777] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2667.681449] Lustre: Skipped 13 previous similar messages [ 2667.683766] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2667.685734] Lustre: Skipped 13 previous similar messages [ 2667.687076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.689986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2667.691698] Lustre: Skipped 13 previous similar messages [ 2667.967186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.275970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.594944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.876219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.159518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.440676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.756644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.051392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.337968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.628192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.927772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.216895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.533751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.816907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.125117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.418368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.723637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.065805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.382603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.681261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.991825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.287154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.646610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.952728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.239509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.529826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.829101] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2675.831477] Lustre: Skipped 26 previous similar messages [ 2675.837046] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2675.840593] Lustre: Skipped 26 previous similar messages [ 2675.842134] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2675.844127] Lustre: Skipped 26 previous similar messages [ 2675.845510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.848751] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2675.851467] Lustre: Skipped 26 previous similar messages [ 2676.149599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.459216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.771890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.092213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.416442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.721394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.011784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.322055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.621499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.972865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.257358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.549242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.827259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.114905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.451990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.760743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.035264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.321229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.604605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.917756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.253964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.536093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.816544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.112356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.397400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.697787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.988506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.299450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.588964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.887411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.173782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.510574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.812210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.097614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.387320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.694820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.005419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.309181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.605974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.929827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.232039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.542215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.844820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.139458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.425929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.728140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.017807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.319701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.596614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.876525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.160384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.473678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.879278] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2691.884099] Lustre: Skipped 52 previous similar messages [ 2691.893598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2691.902501] Lustre: Skipped 52 previous similar messages [ 2691.906474] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2691.908883] Lustre: Skipped 52 previous similar messages [ 2691.910646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.919520] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2691.925859] Lustre: Skipped 52 previous similar messages [ 2692.287364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.648354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.997451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.300480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.604227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.909697] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2693.912549] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2693.914582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.205948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.498444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.785304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.093186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.381284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.671041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.968632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.280200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.565229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.895707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.186463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.495403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.787830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.067114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.371068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.694887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.009447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.312520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.608268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.924718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.236643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.545833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.853910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.173481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.491705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.845166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.219326] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2702.222787] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2702.226883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.603229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.944871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.255242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.572907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.908905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.198752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.507937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.797582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.091456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.387669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.705070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.000444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.300162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.597493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.903920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.232178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.526819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.807418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.101557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.385625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.712058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.004936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.300587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.575286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.863438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.154575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.448587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.733810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.022717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.310462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.607136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.883763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.163195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.443341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.727761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.050171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.346536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.636469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.936232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.230453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.523397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.816766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.120732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.422349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.726650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.058560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.392240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.692305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.018468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.334615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.668375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.981928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.288029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.576610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.895515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.206500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.515846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.820722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.153827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.474526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.784266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.098468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.428771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.730355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.046246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.374826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.701572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.975711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.251427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.548541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.862640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.140121] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2724.142875] Lustre: Skipped 104 previous similar messages [ 2724.147935] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2724.153333] Lustre: Skipped 104 previous similar messages [ 2724.155390] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2724.157792] Lustre: Skipped 104 previous similar messages [ 2724.159504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.162960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2724.164783] Lustre: Skipped 104 previous similar messages [ 2724.443194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.739866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.051829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.355472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.643987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.945692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.252149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.552579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.852096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.159223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.464181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.803583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.085396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.370725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.697713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.019091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.322485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.631831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.949541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.241342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.541671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.843670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.170406] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2731.175418] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2731.175815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.479944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.802071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.157532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.492390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.837357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.225651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.574636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.943595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.299419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.641846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.969904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.289771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.611832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.939361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.245348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.549892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.877496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.199268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.551770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.955911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.324599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.658061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.990546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.321900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.662171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.966680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.271066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.587856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.901095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.189944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.485116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.786508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.092825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.400032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.741502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.042279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.354960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.652868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.988623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.360295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.682255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.985764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.284282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.597280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.932365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.243973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.554520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.862743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.159912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.451088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.797392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.137932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.472052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.808561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.172367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.495933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.834087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.136810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.428929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.764610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.097555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.428972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.780738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.101628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.454093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.790704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.121718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.487430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.822995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.149546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.449140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.802752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.133529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.441370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.765198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.110385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.413877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.721709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.047857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.396422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.695864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.991859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.318935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.692326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.008438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.336147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.660992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.996993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.330461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.679264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.990767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.298274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.603343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.894133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.222147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.534344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.834131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.136142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.427962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.745227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.048139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.337469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.618993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.921711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.209168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.481722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.762702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.070773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.356633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.668325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.975201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.294947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.589699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.899876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.230221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.579509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.909679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.240221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.589965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.938402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.242193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.538245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.832157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.149808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.471530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.769352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.062280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.387142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.704139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.003945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.297590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.588937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.884721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.177603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.471293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.780758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.061583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.348128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.636528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.942564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.262019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.572404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.891152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.206508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.531165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.863405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.167532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.475566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.778845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.081908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.371824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.667470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.947657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.235483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.535838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.850254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.145996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.446324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.753736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.074522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.360335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.648427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.941261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.245402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.541259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.822270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.108932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.405087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.693218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.003453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.318675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.636625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.936089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.242600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.549621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.878961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.196330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.532207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.832069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.121621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.395277] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2788.397250] Lustre: Skipped 203 previous similar messages [ 2788.401399] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2788.404712] Lustre: Skipped 203 previous similar messages [ 2788.406121] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2788.407835] Lustre: Skipped 203 previous similar messages [ 2788.409215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.412241] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2788.413802] Lustre: Skipped 203 previous similar messages [ 2788.683741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.966352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.270369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.567071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.853550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.154488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.466256] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2790.470817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.475519] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2790.771225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.074478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.383569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.720107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.019772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.310867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.607910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.919216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.202508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.487159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.775841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.086380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.365480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.671848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.983670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.295147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.605890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2795.920942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.230431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.554346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2796.860497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.176209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.485491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2797.784585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.069332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.358391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.661486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2798.961994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.257514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.561316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.859323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.173450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.469521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.760668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.069871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.401633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.716720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.005599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.289533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.602938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.899993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.204927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.486381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.794907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.113269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.416728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.739779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.059433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.366024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.675291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.978751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.315769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.624741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.932613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.229863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.560857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.865377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.161901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.465865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.793059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.106531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.414059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.726965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.049963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.340345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.651058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.965858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.274550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.569440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.858550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.157958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.463050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.758122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.050187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.346985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.670887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.987771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.288895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.576668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.887623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.211337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.528066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.824417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.145597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.447512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.738655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.040892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.355787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.645660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.937051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.235281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.567552] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2818.570921] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2818.571765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.880548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.180176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.481255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.797647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.110278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.402571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.693390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.997701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.282121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.573195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.860803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.160778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.455365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.748629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.056811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.370052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.669700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.957224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.253119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.561163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.852373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.151517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.444366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.758900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.070512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.380612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.691326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.012031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.305302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.603806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.912190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.225051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.537501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.850820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.182788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.535225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.820259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.106104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.404267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.721246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.022765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.411964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.755789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.071203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.396819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.729975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.037033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.339792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.647437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.975465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.272392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.598682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.907770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.206501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.501410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.843773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.179745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.502213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.807310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.097773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.398234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.710310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.001502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.314978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.622385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.928707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.246094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.556595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.859355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.190087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.497959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.798927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.118220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.419169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.727196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.052725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.432493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.813209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.160931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.485977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.793630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.103151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.398283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.707887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.015061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.318341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.620291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.914783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.228104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.552465] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2846.557781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.558051] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2846.889910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.220640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.574246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.932853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.230324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.558317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.902241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.205991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.499921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2849.804251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.098130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.367742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.639813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2850.913461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.222298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.557798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2851.945936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.290442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.607286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2852.907225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.213322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.521175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2853.828106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.130970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.444201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2854.763182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.061322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.353522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.646099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2855.960105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.284247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.600929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2856.938305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.277269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.582750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2857.890273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.189266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.508979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2858.814350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.107040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.411841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2859.740623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.047883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.365799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.673427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2860.987230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.273319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.558575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2861.845475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.179584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.495344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2862.803321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.122879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.441515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2863.757113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.063440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.355577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.654718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2864.949884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.235067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.542557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2865.871659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.180047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.492847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2866.798082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.123904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.427654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2867.715586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.018577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.334988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.629134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2868.909227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.208861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.514300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2869.838870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.155430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.497777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2870.817279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.147578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.472941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2871.786638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.099962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.399808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.698634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2872.998204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.304506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.602531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2873.889865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.187806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.527350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2874.896139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.237361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.533243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2875.833450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.163106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.472111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2876.793317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.095553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.410716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2877.729151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.036970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.328642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2878.636459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.013220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.365774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2879.737976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.069992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.469139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2880.822731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.170316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.514353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2881.815982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.148562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.469301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2882.768850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.043615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.332611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.636152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2883.928698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.211321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.527703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2884.836626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.163370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.487579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2885.796484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.083309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.390764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.679826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2886.979438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.276895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.568283] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2887.571883] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2887.573054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2887.874662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.166292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.482195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2888.818866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.120850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.411728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2889.707098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.011374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.286614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.567769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2890.842448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.148473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.441518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2891.739053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.043304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.364900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.677260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2892.997700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.301139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.620512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2893.960832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.265168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.578900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2894.913559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.211496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.511347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2895.827726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.155656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.451234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2896.740434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.017438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.314616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.603524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2897.908322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.197710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.498767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2898.803624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.088052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.380715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.679798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2899.971672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.255291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.537787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2900.841398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.111844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.377789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.655329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2901.946781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.228273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.497214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2902.772788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.061376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.346884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.645325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2903.932870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.245700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.534044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2904.815625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.101219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.386856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.664677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2905.952944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.251960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.562371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2906.850063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.151852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.438178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2907.746875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.057437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.365640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.656834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2908.965857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.262850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.552335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2909.853288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.168202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.467230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2910.767034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.067403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.380404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.668404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2911.960744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.262904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.570698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2912.849405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.130270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.421931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2913.717603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.014066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.319181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.620781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2914.913381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.198359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.495083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2915.791651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.115585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.406210] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2916.408389] Lustre: Skipped 416 previous similar messages [ 2916.413190] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2916.416591] Lustre: Skipped 416 previous similar messages [ 2916.418306] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2916.420234] Lustre: Skipped 416 previous similar messages [ 2916.421587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2916.424631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2916.426248] Lustre: Skipped 416 previous similar messages [ 2916.717964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.006133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.312735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.605170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2917.898918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.196562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.540628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2918.835867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.151366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.475345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2919.787039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.090774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.434465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2920.772088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.097707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.396844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.703133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2921.989688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.322049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.640705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2922.945499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.258834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.601383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2923.928344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.242466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.564360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2924.907187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.211412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.540124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2925.891068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.245336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.578308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2926.877466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.163241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.479238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2927.784556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.095610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.409513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2928.737396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.097427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.469899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2929.785760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.086021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.388226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2930.694737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.004245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.303176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.592822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2931.893703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.213253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.515330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2932.832840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.150111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.493660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2933.806255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.137939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.466501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2934.799111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.144897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.488893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2935.805262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.133135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.484424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2936.853929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.172360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.513120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2937.906914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.245444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2938.657745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.047210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.520079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2939.972815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.436484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2940.910376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.342534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2941.713880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.117136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.523962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2942.926362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.300763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2943.672211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.006549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.366662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2944.706578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.055586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.426570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2945.817829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.133543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.468882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2946.903637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.305636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2947.778718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.139514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.598596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2948.966387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.308166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2949.719217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.060691] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2950.063939] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 2950.065246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.066187] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2950.073662] LustreError: 162442:0:(osp_precreate.c:1355:osp_precreate_thread()) Skipped 1 previous similar message [ 2950.411820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2950.729185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.039560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.358285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2951.695448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.038600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.466123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2952.800254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.198656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.611952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2953.980115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.400725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2954.790799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.137622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.454656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2955.780149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.143528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.504831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2956.889941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.293410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2957.665160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.045033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.423359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2958.833131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.194801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.556316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2959.989386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.382728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2960.738815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.053110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.378311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2961.727575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.062080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.373862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2962.703658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.035144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.346284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2963.693019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.014472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.293721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.610029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2964.899449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.220381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.602357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2965.931758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.210386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.493465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2966.776531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.079951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.377706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.672919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2967.974491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.279687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2968.615087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.010246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.328934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.642482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2969.961793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.295340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2970.668827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.065348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.418894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2971.714807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.014057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.289323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.569209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2972.855200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2973.140311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3059.787115] LustreError: 164015:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 3059.791990] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3059.798645] LustreError: 162442:0:(osp_precreate.c:651:osp_precreate_send()) Skipped 1 previous similar message [ 3128.293560] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3128.297146] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3128.300291] Lustre: Skipped 1 previous similar message [ 3133.411355] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3133.416125] Lustre: Skipped 1 previous similar message [ 3134.216116] Lustre: server umount lustre-OST0000 complete [ 3138.528756] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3138.529475] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3138.532886] Lustre: Skipped 1 previous similar message [ 3141.856954] Lustre: server umount lustre-MDT0000 complete [ 3143.047133] LustreError: 186743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281198 with bad export cookie 1484004787344448288 [ 3143.049551] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3143.052694] LustreError: 186743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3143.169568] Lustre: server umount lustre-MDT0001 complete [ 3146.556618] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3147.675988] Key type lgssc unregistered [ 3147.811438] LNet: 217080:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3147.815267] LNetError: 217080:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3147.823377] LNet: Removed LNI 192.168.201.116@tcp [ 3148.153501] Key type .llcrypt unregistered [ 3148.155553] Key type ._llcrypt unregistered [ 3157.267927] Key type ._llcrypt registered [ 3157.269092] Key type .llcrypt registered [ 3157.305498] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3157.665657] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3157.695549] alg: No test for adler32 (adler32-zlib) [ 3158.546556] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 3158.645750] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 3160.231138] Key type lgssc registered [ 3160.609627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3163.541198] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3163.554081] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3164.764991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3166.413286] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3169.513181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3169.660593] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3171.350450] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3173.240336] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3174.424837] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3177.246181] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3177.386668] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3179.326929] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3181.307989] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3189.671271] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49666 to 0x280000401:49697) [ 3194.598340] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3195.235633] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3195.239885] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3195.242144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3195.247150] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3195.247171] LustreError: 218474:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3196.643954] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3196.645053] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3196.650088] LustreError: Skipped 3 previous similar messages [ 3196.656709] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3199.969667] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3199.970460] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3199.975688] Lustre: Skipped 2 previous similar messages [ 3202.128577] Lustre: server umount lustre-OST0000 complete [ 3215.327084] LustreError: 218474:0:(osp_precreate.c:1263:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3230.287464] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3230.403991] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3232.308160] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3234.252750] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3238.064587] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 3 sec [ 3239.654048] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.661456] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3239.788287] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff95ad44000000.ost_server_uuid 50 [ 3240.458758] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff95ad44000000.ost_server_uuid in FULL state after 0 sec [ 3245.035453] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3245.044108] LustreError: Skipped 1 previous similar message [ 3245.047235] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3245.053616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3248.296226] Lustre: server umount lustre-OST0000 complete [ 3250.011588] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3250.014244] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3250.018866] Lustre: Skipped 1 previous similar message [ 3250.023795] Lustre: Skipped 1 previous similar message [ 3255.265332] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3255.269489] Lustre: Skipped 1 previous similar message [ 3256.192051] Lustre: server umount lustre-MDT0000 complete [ 3258.044448] LustreError: 218434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281313 with bad export cookie 10134861327689382687 [ 3258.049964] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3258.050447] LustreError: 218434:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3258.217381] Lustre: server umount lustre-MDT0001 complete [ 3262.747978] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3264.033092] Key type lgssc unregistered [ 3264.172748] LNet: 223132:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3264.175795] LNetError: 223132:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3264.187341] LNet: Removed LNI 192.168.201.116@tcp [ 3264.538501] Key type .llcrypt unregistered [ 3264.540502] Key type ._llcrypt unregistered [ 3275.761559] Key type ._llcrypt registered [ 3275.763034] Key type .llcrypt registered [ 3275.799627] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3276.124969] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3276.186119] alg: No test for adler32 (adler32-zlib) [ 3277.034388] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 3277.115697] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 3278.695163] Key type lgssc registered [ 3279.095122] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3282.118529] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3282.125351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3283.285300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3284.581293] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3287.475266] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3287.595048] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3288.823669] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3289.783605] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3292.013142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3292.145748] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3293.159600] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49729) [ 3294.435592] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3297.542079] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3297.607569] Lustre: lustre-OST0001: new disk, initializing [ 3297.610230] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3297.644708] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3299.633529] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3301.897784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3301.902101] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3301.933190] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3305.247363] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3308.075264] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3322.337647] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3322.338679] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3322.341960] Lustre: Skipped 3 previous similar messages [ 3322.344321] Lustre: Skipped 2 previous similar messages [ 3325.536592] Lustre: server umount lustre-MDT0000 complete [ 3326.760164] LustreError: 224800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281382 with bad export cookie 1388016245588923328 [ 3326.762844] LustreError: 225536:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 3326.763569] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3326.764784] LustreError: 224800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3326.884657] Lustre: server umount lustre-MDT0001 complete [ 3338.376415] Lustre: server umount lustre-OST0000 complete [ 3348.834806] Lustre: server umount lustre-OST0001 complete [ 3355.792085] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3359.426954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3359.605024] LustreError: 230151:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3359.610056] LustreError: 230151:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3359.624857] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3360.935060] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3364.040838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3365.546222] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3366.594256] Lustre: 231257:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3369.027937] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3369.176735] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3369.178689] Lustre: Skipped 1 previous similar message [ 3370.211032] LustreError: 231612:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3370.214616] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49761) [ 3371.165419] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3374.241217] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3374.314899] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 3376.248270] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 3376.444269] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3379.701837] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3380.938412] Lustre: 233100:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3395.045273] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3395.046926] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3395.047828] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3395.047837] Lustre: Skipped 1 previous similar message [ 3395.050595] Lustre: Skipped 1 previous similar message [ 3396.574181] Lustre: server umount mylustre-MDT0000 complete [ 3398.040461] LustreError: 233102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281453 with bad export cookie 1388016245588926184 [ 3398.046055] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3398.047716] LustreError: 233102:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3398.184300] Lustre: server umount mylustre-MDT0001 complete [ 3410.375257] Lustre: server umount mylustre-OST0000 complete [ 3422.046515] Lustre: server umount mylustre-OST0001 complete [ 3429.244804] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3432.712339] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3432.869214] LustreError: 235389:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3432.892468] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3432.894411] Lustre: Skipped 1 previous similar message [ 3434.270501] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3437.271877] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3438.749704] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3439.738542] Lustre: 236498:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3442.031311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3443.171417] LustreError: 236853:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3443.175673] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49793) [ 3444.207311] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3447.266719] LustreError: 236853:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3447.273104] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 3447.275057] LustreError: 236853:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3447.291834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3448.500250] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 3449.576470] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3452.777939] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3454.011370] Lustre: 238339:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3463.138181] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3463.139713] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3463.140129] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3463.140133] Lustre: Skipped 3 previous similar messages [ 3463.143708] Lustre: Skipped 1 previous similar message [ 3467.164815] Lustre: server umount tfs-MDT0000 complete [ 3468.257660] LustreError: 235408:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3468.264026] LustreError: 235408:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3468.430238] LustreError: 236877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281523 with bad export cookie 1388016245588928361 [ 3468.433836] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3468.435833] LustreError: 236877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3468.552338] Lustre: server umount tfs-MDT0001 complete [ 3480.583424] Lustre: server umount tfs-OST0000 complete [ 3492.267349] Lustre: server umount tfs-OST0001 complete [ 3500.909867] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3504.790146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3505.007630] LustreError: 240570:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3505.040462] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3505.043128] Lustre: Skipped 3 previous similar messages [ 3506.480748] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3509.770897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3511.400787] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3512.559904] Lustre: 241677:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3515.079583] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3517.344127] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3520.361894] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 3520.600356] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3522.545544] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 3523.151874] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3529.708843] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49699 to 0x280000401:49825) [ 3537.717696] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3539.386228] Lustre: 243543:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3547.033937] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:06:42 (1778281602) [ 3560.418830] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3560.419932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3560.423148] Lustre: Skipped 4 previous similar messages [ 3560.427183] Lustre: Skipped 6 previous similar messages [ 3565.537048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3565.539151] Lustre: Skipped 3 previous similar messages [ 3573.215087] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3573.275088] Lustre: server umount lustre-MDT0000 complete [ 3574.480062] LustreError: 240545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281629 with bad export cookie 1388016245588930279 [ 3574.483108] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3574.484346] LustreError: 240545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3575.776140] LustreError: 243298:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3575.776580] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3575.777435] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3575.777438] Lustre: Skipped 4 previous similar messages [ 3575.783064] LustreError: 243298:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3575.787618] Lustre: Skipped 2 previous similar messages [ 3580.648949] Lustre: server umount lustre-MDT0001 complete [ 3596.255103] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3596.289388] Lustre: server umount lustre-OST0000 complete [ 3603.736100] Lustre: server umount lustre-OST0001 complete [ 3605.853752] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 3609.127486] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3620.835863] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 3620.835863] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3620.835863] [ 3620.841495] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3620.923945] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3620.936295] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3620.971222] Lustre: lustre-MDT0000: new disk, initializing [ 3620.994919] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3620.997708] Lustre: Skipped 3 previous similar messages [ 3621.003656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3622.307507] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3626.484523] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 3626.484523] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 3626.484523] [ 3626.507057] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 3626.542094] Lustre: 246691:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3626.563978] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3626.565395] Lustre: Skipped 1 previous similar message [ 3626.597347] Lustre: lustre-MDT0001: new disk, initializing [ 3626.640200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3626.644257] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3627.905974] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3632.031596] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3632.134915] Lustre: lustre-OST0000: new disk, initializing [ 3632.136965] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3633.972709] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3634.168085] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3634.170746] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3634.204649] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3638.136827] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3638.202471] Lustre: lustre-OST0001: new disk, initializing [ 3638.204609] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3639.935983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3639.939840] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3639.971786] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3640.235720] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3644.456690] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:08:19 (1778281699) [ 3647.919202] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:08:23 (1778281703) [ 3659.745290] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3659.745383] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3659.748403] LustreError: Skipped 1 previous similar message [ 3659.754106] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3659.757729] Lustre: Skipped 4 previous similar messages [ 3665.407895] Lustre: server umount lustre-OST0000 complete [ 3665.891781] LustreError: 248224:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3665.896420] LustreError: 248224:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3667.935737] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3672.927057] Lustre: server umount lustre-MDT0000 complete [ 3674.153219] LustreError: 246682:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281729 with bad export cookie 1388016245588932057 [ 3674.156516] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3674.157345] LustreError: 246682:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3676.128435] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3676.132272] Lustre: Skipped 4 previous similar messages [ 3680.304418] Lustre: server umount lustre-MDT0001 complete [ 3684.120619] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3702.239103] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3702.297217] Lustre: server umount lustre-OST0001 complete [ 3704.295683] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 3710.822298] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3723.660617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3723.749342] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3723.761588] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3723.798795] Lustre: lustre-MDT0000: new disk, initializing [ 3723.829342] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3725.138624] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3729.422746] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3729.467737] Lustre: 253548:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3729.570099] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3730.995109] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3734.236920] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3735.483114] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3737.918340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3739.457583] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3739.499793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3740.187037] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3743.609427] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3749.857503] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3749.858081] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3749.863061] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3749.864722] LustreError: Skipped 1 previous similar message [ 3749.867376] Lustre: Skipped 8 previous similar messages [ 3752.146430] Lustre: server umount lustre-OST0000 complete [ 3759.643719] Lustre: server umount lustre-MDT0000 complete [ 3760.861173] LustreError: 253540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281816 with bad export cookie 1388016245588933527 [ 3760.863683] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3760.864711] LustreError: 253540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3760.982211] Lustre: server umount lustre-MDT0001 complete [ 3764.438797] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3765.505392] Key type lgssc unregistered [ 3765.642415] LNet: 256919:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3765.644717] LNetError: 256919:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3765.653292] LNet: Removed LNI 192.168.201.116@tcp [ 3765.944162] Key type .llcrypt unregistered [ 3765.945771] Key type ._llcrypt unregistered [ 3775.574644] Key type ._llcrypt registered [ 3775.575562] Key type .llcrypt registered [ 3775.610657] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3775.908703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3775.930537] alg: No test for adler32 (adler32-zlib) [ 3776.786736] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 3776.881954] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 3778.463314] Key type lgssc registered [ 3778.832523] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3781.524755] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3781.531855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3782.687086] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3784.048132] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3786.995631] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3787.152047] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3788.556213] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3790.556171] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3791.851299] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3794.286225] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3794.410138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3796.377922] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3798.197702] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3798.541439] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3802.592329] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3802.594967] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3802.598990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3805.324632] Lustre: server umount lustre-OST0000 complete [ 3807.712508] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3807.712919] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3807.713610] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3807.717949] Lustre: Skipped 1 previous similar message [ 3812.758654] Lustre: server umount lustre-MDT0000 complete [ 3812.832221] LustreError: 258339:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3812.838053] LustreError: 258339:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3814.063259] LustreError: 258319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778281869 with bad export cookie 11985768776296668089 [ 3814.065111] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3814.067959] LustreError: 258319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3814.178042] Lustre: server umount lustre-MDT0001 complete [ 3817.755914] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3818.743936] Key type lgssc unregistered [ 3818.875388] LNet: 261569:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3818.878210] LNetError: 261569:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3818.890342] LNet: Removed LNI 192.168.201.116@tcp [ 3819.178841] Key type .llcrypt unregistered [ 3819.179699] Key type ._llcrypt unregistered [ 3825.967233] Key type ._llcrypt registered [ 3825.968280] Key type .llcrypt registered [ 3826.004886] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3826.368642] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3826.384795] alg: No test for adler32 (adler32-zlib) [ 3827.242074] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 3827.328517] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 3828.911210] Key type lgssc registered [ 3829.289391] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3832.436747] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3832.442381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3847.903261] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3853.023180] LustreError: 262723:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.116@tcp: failed processing log, type 4: rc = -110 [ 3879.647156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3881.530148] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 3883.243207] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3883.963287] Lustre: server umount lustre-OST0000 complete [ 3889.303773] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 3890.385241] Key type lgssc unregistered [ 3890.525373] LNet: 264222:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3890.527721] LNetError: 264222:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3890.534342] LNet: Removed LNI 192.168.201.116@tcp [ 3890.817515] Key type .llcrypt unregistered [ 3890.818914] Key type ._llcrypt unregistered [ 3903.996930] Key type ._llcrypt registered [ 3903.998178] Key type .llcrypt registered [ 3904.039789] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3904.488433] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3904.496344] alg: No test for adler32 (adler32-zlib) [ 3905.358291] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 3905.440383] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 3907.015136] Key type lgssc registered [ 3907.357703] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3911.616185] loop0: detected capacity change from 0 to 204800000 [ 3911.687095] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3911.692212] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3911.807193] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.147634] loop0: detected capacity change from 0 to 204800000 [ 3912.207303] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.213159] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.331524] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.703789] loop0: detected capacity change from 0 to 204800000 [ 3912.769972] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.774343] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3912.814251] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3913.191194] loop0: detected capacity change from 0 to 204800000 [ 3913.287852] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 3913.669384] loop0: detected capacity change from 0 to 204800000 [ 3913.683648] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3915.010517] loop0: detected capacity change from 0 to 204800000 [ 3915.020289] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3916.317196] loop0: detected capacity change from 0 to 204800000 [ 3916.330335] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3917.590897] loop0: detected capacity change from 0 to 204800000 [ 3917.609309] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3918.884096] loop0: detected capacity change from 0 to 204800000 [ 3918.912685] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3918.922314] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3918.957407] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3918.961479] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 3918.964914] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 3918.968847] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 3918.972740] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 3918.975850] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3918.979500] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3918.983351] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3918.987941] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 3918.991441] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 3918.994529] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 3918.998544] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 3919.001740] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 3919.006204] Lustre: 267593:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 3921.522613] Lustre: server umount MGS complete [ 3921.810264] loop0: detected capacity change from 0 to 204800000 [ 3921.837787] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3921.994995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3922.010119] Lustre: lustre-MDT0000: Aborting client recovery [ 3922.306480] loop1: detected capacity change from 0 to 204800000 [ 3922.340544] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3922.381497] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3922.384901] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 3922.390134] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3922.394519] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3922.397694] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3922.401299] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 3922.406644] Lustre: 268025:0:(osd_scrub.c:1829:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 3922.414178] Lustre: Found index 1 for lustre-MDT0001, updating log [ 3922.512775] Lustre: lustre-MDT0001: Aborting client recovery [ 3922.828418] loop2: detected capacity change from 0 to 204800000 [ 3922.857609] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3922.892179] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3922.896675] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3922.902938] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 3922.907261] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 3922.912360] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 3922.917891] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3922.921955] Lustre: 268107:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3922.969880] Lustre: Found index 0 for lustre-OST0000, updating log [ 3922.999345] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3923.001555] Lustre: Skipped 1 previous similar message [ 3923.009181] Lustre: lustre-OST0000: Aborting recovery [ 3923.315202] loop3: detected capacity change from 0 to 204800000 [ 3923.344170] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3923.376374] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 3923.380547] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 3923.384292] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 3923.388107] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 3923.391761] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 3923.395174] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 3923.400618] Lustre: 268191:0:(osd_scrub.c:1829:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 3928.552987] LustreError: 267969:0:(lod_dev.c:510:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 3931.618504] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3931.619181] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 3931.619207] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 3931.621795] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 3931.623940] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 3931.627874] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 3933.878183] LustreError: 267935:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 195us: rc = -2 [ 3946.976697] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3946.977214] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3946.980964] Lustre: Skipped 3 previous similar messages [ 3946.982808] Lustre: Skipped 2 previous similar messages [ 3951.205978] Lustre: server umount lustre-MDT0000 complete [ 3951.521785] LustreError: 267964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282006 with bad export cookie 3213900391023506343 [ 3951.523870] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3951.525872] LustreError: 267964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3952.095612] LustreError: 268421:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3952.095826] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3952.096119] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3952.096125] Lustre: Skipped 1 previous similar message [ 3952.100040] LustreError: 268421:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3952.103455] Lustre: Skipped 1 previous similar message [ 3956.127582] LustreError: 267934:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3956.127727] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3956.132242] LustreError: 267934:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3956.134405] Lustre: Skipped 2 previous similar messages [ 3957.766739] Lustre: server umount lustre-MDT0001 complete [ 3964.170811] Lustre: server umount lustre-OST0000 complete [ 3970.586131] Lustre: server umount lustre-OST0001 complete [ 3973.176922] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:13:48 (1778282028) [ 3981.538042] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 3984.759374] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 3996.077216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3996.170171] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3996.181561] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3996.214549] Lustre: lustre-MDT0000: new disk, initializing [ 3996.238072] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3996.239545] Lustre: Skipped 1 previous similar message [ 3996.244147] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3997.445230] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4001.637541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4001.682986] Lustre: 272230:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4001.707189] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4001.709534] Lustre: Skipped 1 previous similar message [ 4001.750889] Lustre: lustre-MDT0001: new disk, initializing [ 4001.785106] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4001.797837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4001.802064] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4003.179312] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4006.221488] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4007.351292] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4009.517396] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4009.608865] Lustre: lustre-OST0000: new disk, initializing [ 4009.610483] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4009.635871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4011.610051] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4014.827285] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4016.119052] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4016.122993] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4016.153355] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4022.734960] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4025.230940] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4033.453387] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4077.536300] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4077.537993] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4077.538673] LustreError: Skipped 1 previous similar message [ 4077.542196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4077.544926] Lustre: Skipped 1 previous similar message [ 4082.657575] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4082.659810] Lustre: Skipped 1 previous similar message [ 4087.263110] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4087.312664] Lustre: server umount lustre-OST0000 complete [ 4088.799713] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4088.801877] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4088.805266] Lustre: Skipped 1 previous similar message [ 4092.896129] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4092.896406] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4092.898973] Lustre: Skipped 1 previous similar message [ 4094.619201] Lustre: server umount lustre-MDT0000 complete [ 4095.825089] LustreError: 273007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282151 with bad export cookie 3213900391023509059 [ 4095.828130] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4095.829468] LustreError: 273007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4095.930898] Lustre: server umount lustre-MDT0001 complete [ 4098.758423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4100.183663] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4102.013958] Lustre: server umount MGS complete [ 4106.127980] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4106.267919] LustreError: 277746:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4106.274151] LustreError: 277759:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4106.295573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4107.572941] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4110.457025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4111.921789] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4113.763577] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4115.103956] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4117.505562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4119.761805] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4121.758411] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4151.777856] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4151.782146] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4151.782410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4151.786085] Lustre: Skipped 1 previous similar message [ 4157.586772] Lustre: server umount lustre-OST0000 complete [ 4162.018562] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4173.279095] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4173.331947] Lustre: server umount lustre-MDT0000 complete [ 4174.591136] LustreError: 278465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282229 with bad export cookie 3213900391023510886 [ 4174.593955] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4174.595253] LustreError: 278465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4174.704936] Lustre: server umount lustre-MDT0001 complete [ 4178.082131] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 4179.028287] Key type lgssc unregistered [ 4179.157345] LNet: 281358:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4179.160634] LNetError: 281358:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4179.167302] LNet: Removed LNI 192.168.201.116@tcp [ 4179.435418] Key type .llcrypt unregistered [ 4179.436734] Key type ._llcrypt unregistered [ 4190.937246] Key type ._llcrypt registered [ 4190.938807] Key type .llcrypt registered [ 4190.974445] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 4196.893524] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 4197.334729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4197.341549] alg: No test for adler32 (adler32-zlib) [ 4198.196468] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 4198.280214] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 4199.863194] Key type lgssc registered [ 4200.205723] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4211.073226] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4211.080191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4212.162781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4212.172446] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4212.206196] Lustre: lustre-MDT0000: new disk, initializing [ 4212.229247] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4212.235583] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4213.508843] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4217.572436] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4217.609541] Lustre: 285255:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4217.633274] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4217.636218] Lustre: Skipped 1 previous similar message [ 4217.672835] Lustre: lustre-MDT0001: new disk, initializing [ 4217.703147] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4217.715348] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4217.718910] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4219.003073] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4222.142808] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4223.360232] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4225.598889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4225.700345] Lustre: lustre-OST0000: new disk, initializing [ 4225.703043] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4225.732488] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4227.639546] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4227.642620] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4227.675815] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4227.723052] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4230.863300] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4234.313598] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4239.385165] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4245.211399] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4283.873629] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4283.874195] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4283.876319] LustreError: Skipped 1 previous similar message [ 4283.880568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4283.883049] Lustre: Skipped 1 previous similar message [ 4287.566107] Lustre: server umount lustre-OST0000 complete [ 4288.992610] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4288.996527] Lustre: Skipped 1 previous similar message [ 4288.998625] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4289.000265] Lustre: Skipped 1 previous similar message [ 4294.112060] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4294.113972] Lustre: Skipped 1 previous similar message [ 4299.232031] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4299.234994] Lustre: Skipped 1 previous similar message [ 4303.327131] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4303.390566] Lustre: server umount lustre-MDT0000 complete [ 4304.665236] LustreError: 286994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282360 with bad export cookie 9007168458798236950 [ 4304.668217] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4304.669528] LustreError: 286994:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4304.782682] Lustre: server umount lustre-MDT0001 complete [ 4307.705756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4309.052234] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4310.685819] Lustre: server umount MGS complete [ 4314.481299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4314.621303] LustreError: 290769:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4314.627538] LustreError: 290769:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 4314.627566] LustreError: 290782:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4314.652265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4315.980108] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4318.934552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4319.071597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4320.451424] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4322.334504] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4323.626520] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4325.880503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4327.733109] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4329.381291] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4365.280126] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4365.282698] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4365.286714] Lustre: Skipped 1 previous similar message [ 4365.289186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4366.306551] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4368.779414] Lustre: server umount lustre-OST0000 complete [ 4370.399849] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4370.402170] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4370.405872] Lustre: Skipped 1 previous similar message [ 4376.218419] Lustre: server umount lustre-MDT0000 complete [ 4376.543827] LustreError: 290770:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4376.551835] LustreError: 290770:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4377.415149] LustreError: 290751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282432 with bad export cookie 9007168458798238763 [ 4377.417049] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4377.419355] LustreError: 290751:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4377.538492] Lustre: server umount lustre-MDT0001 complete [ 4381.072758] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 4382.111356] Key type lgssc unregistered [ 4382.237377] LNet: 294377:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4382.239711] LNetError: 294377:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4382.247492] LNet: Removed LNI 192.168.201.116@tcp [ 4382.564604] Key type .llcrypt unregistered [ 4382.565794] Key type ._llcrypt unregistered [ 4389.021897] Key type ._llcrypt registered [ 4389.023525] Key type .llcrypt registered [ 4389.069572] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 4395.051233] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 4395.451735] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4395.479354] alg: No test for adler32 (adler32-zlib) [ 4396.334475] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 4396.422221] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 4397.999121] Key type lgssc registered [ 4398.397165] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4406.239496] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 4411.121222] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4411.126959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4412.217900] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4412.228883] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4412.266871] Lustre: lustre-MDT0000: new disk, initializing [ 4412.295423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4412.302677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4413.738161] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4417.876104] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4417.972659] Lustre: lustre-OST0000: new disk, initializing [ 4417.974456] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4417.977183] Lustre: Skipped 1 previous similar message [ 4418.000994] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4418.388729] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4418.391213] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4418.399592] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4419.848563] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4874.213532] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4874.223851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4877.674950] Lustre: server umount lustre-MDT0000 complete [ 4880.303701] LustreError: 297535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778282935 with bad export cookie 13608759572331010480 [ 4880.309191] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4880.312765] LustreError: 297535:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4880.397159] Lustre: server umount lustre-OST0000 complete [ 4897.877997] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 4904.832203] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 4932.726626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4932.908528] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4932.940648] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4933.010639] Lustre: lustre-MDT0000: new disk, initializing [ 4933.074185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4933.089525] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4936.187348] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4944.826452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4944.945535] Lustre: 302555:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4944.989465] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4944.992510] Lustre: Skipped 1 previous similar message [ 4945.065708] Lustre: lustre-MDT0001: new disk, initializing [ 4945.158686] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4945.202538] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4945.210481] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4948.631274] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4955.363173] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4958.644714] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4964.287447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4964.469278] Lustre: lustre-OST0000: new disk, initializing [ 4964.472674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4964.552111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4969.686922] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 4970.551195] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4970.568615] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4970.681468] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4976.425572] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4980.167968] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4980.315724] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4983.719310] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4983.883979] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4986.337108] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4986.347077] Lustre: Skipped 1 previous similar message [ 4986.355413] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4986.358096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4986.382192] Lustre: Skipped 1 previous similar message [ 4991.470049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4991.478672] Lustre: Skipped 1 previous similar message [ 4991.689129] Lustre: server umount lustre-OST0000 complete [ 4996.581063] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4996.581393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4996.585555] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4996.596989] Lustre: Skipped 1 previous similar message [ 5000.717385] Lustre: server umount lustre-MDT0000 complete [ 5003.774903] LustreError: 302545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283059 with bad export cookie 13608759572333951264 [ 5003.788253] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5004.031579] Lustre: server umount lustre-MDT0001 complete [ 5006.681869] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5008.181380] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:31:02 (1778283062) [ 5013.581525] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5013.978866] LustreError: 306457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5014.002971] LustreError: 306457:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5014.036549] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5017.370648] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5019.105260] LustreError: 306457:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5019.115335] LustreError: 306457:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5023.624131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5027.178200] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5031.402869] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5033.854681] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5038.832349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5039.084953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5039.087957] Lustre: Skipped 1 previous similar message [ 5043.598148] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5048.194244] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5053.619995] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5053.742531] Lustre: lustre-OST0001: new disk, initializing [ 5053.745960] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5058.617679] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5063.192351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5063.201152] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5063.234373] LustreError: 309529:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5064.292159] LustreError: 309529:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5064.303575] LustreError: 309529:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 5065.244714] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5065.317378] LustreError: 309539:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5065.368743] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5067.363579] LustreError: 309539:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5067.377483] LustreError: 309539:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5068.031816] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff95ad787a4800.ost_server_uuid 50 [ 5068.938278] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff95ad787a4800.ost_server_uuid in FULL state after 0 sec [ 5071.200319] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff95ad787a4800.ost_server_uuid 50 [ 5071.523224] LustreError: 309539:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 5071.529201] LustreError: 309539:0:(osp_precreate.c:969:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [ 5071.976308] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff95ad787a4800.ost_server_uuid in FULL state after 0 sec [ 5073.592633] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5088.481750] Lustre: lustre-OST0001: Not available for connect from 192.168.201.16@tcp (stopping) [ 5088.484987] Lustre: Skipped 1 previous similar message [ 5088.743102] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5091.110372] Lustre: server umount lustre-OST0001 complete [ 5093.859257] LustreError: 309859:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5093.883158] LustreError: 309859:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5095.392231] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5095.397046] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5095.410261] Lustre: Skipped 1 previous similar message [ 5098.976744] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5098.978735] LustreError: 308048:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5098.981469] Lustre: Skipped 5 previous similar messages [ 5098.997137] LustreError: 308048:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5100.426706] Lustre: server umount lustre-OST0000 complete [ 5104.096580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5104.106162] Lustre: Skipped 2 previous similar messages [ 5109.054055] Lustre: server umount lustre-MDT0000 complete [ 5109.216123] LustreError: 306458:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5109.223663] LustreError: 306458:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5111.342807] LustreError: 306439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283166 with bad export cookie 13608759572333952146 [ 5111.343349] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5111.350651] LustreError: 306439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 5111.657949] Lustre: server umount lustre-MDT0001 complete [ 5118.407244] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 5120.170501] Key type lgssc unregistered [ 5120.361615] LNet: 311393:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5120.369114] LNetError: 311393:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5120.384613] LNet: Removed LNI 192.168.201.116@tcp [ 5120.945153] Key type .llcrypt unregistered [ 5120.946633] Key type ._llcrypt unregistered [ 5135.698664] Key type ._llcrypt registered [ 5135.700122] Key type .llcrypt registered [ 5135.749740] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5136.138688] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5136.148826] alg: No test for adler32 (adler32-zlib) [ 5137.049108] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 5137.150978] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 5138.751258] Key type lgssc registered [ 5139.320627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5144.049317] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5144.068661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5145.412311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5147.912192] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5150.895586] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5154.796423] LustreError: 312773:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5154.807547] LustreError: 312773:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5155.468834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5155.771778] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5158.983344] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5165.678769] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5165.932047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5166.959953] LustreError: 314321:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5169.800924] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5170.023848] LustreError: 314425:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5170.030314] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5170.031476] LustreError: 314425:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5173.196357] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5175.269720] LustreError: 314425:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5175.285163] LustreError: 314425:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5177.482649] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5177.659439] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5179.536179] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5181.240726] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5184.737322] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 5189.236441] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5189.397097] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5280.228499] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5280.237317] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5280.239441] Lustre: Skipped 1 previous similar message [ 5284.650056] Lustre: server umount lustre-OST0001 complete [ 5285.346747] LustreError: 314322:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5285.357937] LustreError: 314322:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5289.954528] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5289.967583] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5289.983568] Lustre: Skipped 1 previous similar message [ 5289.990713] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5295.585124] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5295.587209] LustreError: 314322:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5295.589844] Lustre: Skipped 2 previous similar messages [ 5295.613183] LustreError: 314322:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5300.705548] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5300.709620] Lustre: Skipped 2 previous similar messages [ 5302.241364] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5302.348303] Lustre: server umount lustre-OST0000 complete [ 5304.288680] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5304.294607] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5304.301713] Lustre: Skipped 1 previous similar message [ 5305.824855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5305.828302] Lustre: Skipped 2 previous similar messages [ 5310.477427] Lustre: server umount lustre-MDT0000 complete [ 5312.816740] LustreError: 312754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283368 with bad export cookie 11262887600907261534 [ 5312.821075] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5312.823273] LustreError: 312754:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5313.143409] Lustre: server umount lustre-MDT0001 complete [ 5320.107964] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 5322.074634] Key type lgssc unregistered [ 5322.326904] LNet: 317344:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5322.332333] LNetError: 317344:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5322.343145] LNet: Removed LNI 192.168.201.116@tcp [ 5322.878541] Key type .llcrypt unregistered [ 5322.880586] Key type ._llcrypt unregistered [ 5353.395392] Key type ._llcrypt registered [ 5353.397605] Key type .llcrypt registered [ 5353.503725] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5354.194278] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5354.278988] alg: No test for adler32 (adler32-zlib) [ 5355.273816] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 5355.465730] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 5357.127459] Key type lgssc registered [ 5358.049143] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5364.798939] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5364.812455] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5366.150886] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5369.085266] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5375.361064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5375.653926] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5378.513480] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5380.750763] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5386.259290] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5386.531201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5390.682648] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5391.653778] LustreError: 321469:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5391.668235] LustreError: 321469:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5391.670207] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5396.433962] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5396.597746] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5399.691643] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5400.717827] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5410.447453] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5418.651817] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5441.009679] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5441.030786] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5441.045244] Lustre: Skipped 2 previous similar messages [ 5444.004761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5447.746338] Lustre: server umount lustre-MDT0000 complete [ 5449.186626] LustreError: 319595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5449.235583] LustreError: 319595:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5452.654172] LustreError: 322550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283508 with bad export cookie 7395186724677877014 [ 5452.667468] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5452.671606] LustreError: 322550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5453.503659] Lustre: server umount lustre-MDT0001 complete [ 5467.445398] Lustre: server umount lustre-OST0000 complete [ 5470.495120] Lustre: 318863:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1778283509/real 1778283509] req@ffff9916562bbb80 x1864665304989312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1778283525 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5470.530808] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5470.543279] Lustre: Skipped 3 previous similar messages [ 5471.642729] Lustre: server umount lustre-OST0001 complete [ 5487.100892] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5496.344358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5496.741734] LustreError: 324798:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5496.807943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5500.952888] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5501.921234] LustreError: 324799:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5507.042583] LustreError: 324798:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5508.168293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5508.556626] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5512.500586] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5514.966804] Lustre: 325906:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5521.209964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5522.659536] LustreError: 326259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5522.678878] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5527.295671] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5535.577162] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5535.950641] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5535.959393] Lustre: Skipped 1 previous similar message [ 5537.308042] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5542.334261] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5549.120816] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5553.097104] Lustre: 327749:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5564.899081] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5564.915681] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5564.927556] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5564.935506] Lustre: Skipped 3 previous similar messages [ 5566.946714] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5566.953995] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5566.962991] Lustre: Skipped 2 previous similar messages [ 5568.667327] Lustre: server umount lustre-MDT0000 complete [ 5572.068333] LustreError: 325515:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5572.086087] LustreError: 325515:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 5572.095415] LustreError: 325525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283627 with bad export cookie 7395186724677878708 [ 5572.100567] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5572.114284] LustreError: 325525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5572.520606] Lustre: server umount lustre-MDT0001 complete [ 5586.473901] Lustre: server umount lustre-OST0000 complete [ 5600.334874] Lustre: server umount lustre-OST0001 complete [ 5603.765916] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5605.329244] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5606.755783] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 19:41:01 (1778283661) [ 5626.260995] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5633.461280] loop0: detected capacity change from 0 to 1099511627776 [ 5633.934581] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5633.942804] *** VALIDATE xfs *** [ 5633.950735] XFS (loop0): Mounting V5 Filesystem [ 5633.979383] XFS (loop0): Ending clean mount [ 5635.256343] loop1: detected capacity change from 0 to 18691697672192 [ 5713.591425] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 5722.611208] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5751.543589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5751.773251] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5751.800831] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5751.860631] Lustre: lustre-MDT0000: new disk, initializing [ 5751.934212] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5751.952156] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5755.289844] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5763.307693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5763.383093] Lustre: 332834:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5763.415301] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5763.417560] Lustre: Skipped 1 previous similar message [ 5763.460151] Lustre: lustre-MDT0001: new disk, initializing [ 5763.538372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5763.544970] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5766.243514] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5772.246527] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5775.012812] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5779.787706] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5780.026957] Lustre: lustre-OST0000: new disk, initializing [ 5780.035232] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5782.087685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5782.092500] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5782.186308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5784.888387] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5791.088769] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5794.531972] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5794.740798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5798.123625] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5798.298286] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5802.983664] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5802.985909] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5802.993701] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5802.993711] Lustre: Skipped 3 previous similar messages [ 5803.006915] LustreError: Skipped 1 previous similar message [ 5805.877915] Lustre: server umount lustre-OST0000 complete [ 5809.632636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5813.220910] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5813.221291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5813.233447] Lustre: Skipped 2 previous similar messages [ 5813.245341] Lustre: Skipped 2 previous similar messages [ 5814.819360] Lustre: server umount lustre-MDT0000 complete [ 5817.754132] LustreError: 332826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283873 with bad export cookie 7395186724677880269 [ 5817.761828] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5817.766182] LustreError: 332826:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5818.016187] Lustre: server umount lustre-MDT0001 complete [ 5824.402435] XFS (loop0): Unmounting Filesystem [ 5828.481247] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 19:44:42 (1778283882) [ 5834.298090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5834.725674] LustreError: 336982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5834.785853] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5834.788873] Lustre: Skipped 2 previous similar messages [ 5838.360195] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5844.885160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5848.283673] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5852.911323] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5856.490530] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5861.714228] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5865.532457] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5869.775669] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5890.529975] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5890.538490] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5890.544222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5890.552205] Lustre: Skipped 1 previous similar message [ 5890.563186] Lustre: Skipped 1 previous similar message [ 5893.322941] Lustre: server umount lustre-OST0000 complete [ 5903.875062] Lustre: server umount lustre-MDT0000 complete [ 5905.889665] LustreError: 338055:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5905.909496] LustreError: 338055:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 5906.492349] LustreError: 336964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778283961 with bad export cookie 7395186724677881151 [ 5906.497236] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5906.500846] LustreError: 336964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5906.777866] Lustre: server umount lustre-MDT0001 complete [ 5914.444615] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 5916.596388] Key type lgssc unregistered [ 5916.843532] LNet: 340278:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5916.854583] LNetError: 340278:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5916.866717] LNet: Removed LNI 192.168.201.116@tcp [ 5917.546157] Key type .llcrypt unregistered [ 5917.550464] Key type ._llcrypt unregistered [ 5938.386371] Key type ._llcrypt registered [ 5938.387498] Key type .llcrypt registered [ 5938.483162] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 5939.367695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5939.410286] alg: No test for adler32 (adler32-zlib) [ 5940.415678] Lustre: Lustre: Build Version: 2.17.52_6_gcfb9284 [ 5940.569709] LNet: Added LNI 192.168.201.116@tcp [8/256/0/180] [ 5942.183173] Key type lgssc registered [ 5942.997888] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5948.613537] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5948.630938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5949.947118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5953.204534] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5959.421039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5959.794229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5962.744971] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5966.909252] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5969.981593] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5974.390343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5974.592407] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5978.043899] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 5981.463272] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5991.904181] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5991.911338] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5991.920633] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5994.209756] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.16@tcp (stopping) [ 5995.488583] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5995.491764] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5997.247967] Lustre: server umount lustre-MDT0001 complete [ 5999.312430] LustreError: 341805:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5999.337543] LustreError: 341805:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6000.609565] LustreError: 341804:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6000.635738] LustreError: 341804:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 6004.447408] LustreError: 342920:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6005.565150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6005.646448] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6005.650838] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6005.671536] Lustre: 341798:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6005.893213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6008.706540] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 6011.371580] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6013.952934] LustreError: 345075:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6013.962518] LustreError: 345075:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6013.968681] LustreError: 341800:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6014.015967] LustreError: 345081:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.201.116@tcp: cfg command failed: rc = -17 [ 6014.020906] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 6014.020906] [ 6311.660765] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.16@tcp (stopping) [ 6311.667783] Lustre: Skipped 1 previous similar message [ 6313.441095] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6313.443617] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6313.447751] Lustre: Skipped 1 previous similar message [ 6316.746427] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.16@tcp (stopping) [ 6316.751131] Lustre: Skipped 2 previous similar messages [ 6317.754515] Lustre: server umount lustre-MDT0001 complete [ 6318.561424] LustreError: 342519:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6318.569229] LustreError: 342519:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 6323.680982] LustreError: 341808:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6323.691876] LustreError: 341808:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6323.747611] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6323.814450] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6323.817989] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6323.842858] Lustre: 341797:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6324.059398] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6326.448476] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 6329.320209] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6332.911721] LustreError: 355577:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6332.917050] LustreError: 355577:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6332.922722] LustreError: 341800:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6332.925556] LustreError: 341800:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6332.959191] LustreError: 355584:0:(obd_config.c:760:class_setup()) Device 8 already setup (type osp) [ 6332.962428] LustreError: 355584:0:(obd_config.c:864:class_detach()) OBD device 8 still set up [ 6332.965655] LustreError: 355584:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.201.116@tcp: cfg command failed: rc = -17 [ 6332.969823] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.116@tcp [ 6332.969823] [ 6631.395195] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6631.397261] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6631.402032] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6631.407909] Lustre: Skipped 1 previous similar message [ 6644.191115] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6644.271291] Lustre: server umount lustre-MDT0001 complete [ 6644.425138] LustreError: 341804:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.16@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6649.367973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6649.412965] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6649.415670] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6649.433276] Lustre: 341797:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6649.581731] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6651.526667] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 6654.951281] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6659.559781] LustreError: 367035:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6659.564318] LustreError: 367035:0:(tgt_mount.c:666:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6659.568882] LustreError: 341800:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6659.571785] LustreError: 341800:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6659.599794] LustreError: 367038:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6659.606315] LustreError: 367038:0:(obd_config.c:2101:class_config_llog_handler()) MGC192.168.201.116@tcp: cfg command failed: rc = -114 [ 6659.611184] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6659.611184] [ 6967.265682] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6967.266413] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6967.270374] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6967.270381] Lustre: Skipped 6 previous similar messages [ 6967.273819] Lustre: Skipped 3 previous similar messages [ 6973.283255] Lustre: server umount lustre-MDT0000 complete [ 6974.661221] LustreError: 341789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778285030 with bad export cookie 12962769013366905986 [ 6974.664554] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6974.665163] LustreError: 341789:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6977.504041] LustreError: 350601:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6977.504307] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6977.510116] LustreError: 350601:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 6980.859723] Lustre: server umount lustre-MDT0001 complete [ 6982.359242] Lustre: server umount lustre-OST0000 complete [ 6985.817464] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 6989.549668] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 7002.752530] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:04:17 (1778285057) [ 7005.506067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7005.612682] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7005.625583] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7005.664864] Lustre: lustre-MDT0000: new disk, initializing [ 7005.704848] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7005.714699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7007.208757] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 7011.716800] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7011.759975] Lustre: 380674:0:(mgs_llog.c:1348:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7011.783270] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7011.785945] Lustre: Skipped 1 previous similar message [ 7011.822592] Lustre: lustre-MDT0001: new disk, initializing [ 7011.858540] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7011.873380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7011.877685] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7013.321900] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 7016.565680] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7017.901169] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7020.208497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7020.314596] Lustre: lustre-OST0000: new disk, initializing [ 7020.316541] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7020.345323] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7021.370260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7021.373942] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7021.399624] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7022.332481] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_default_debug -1 all [ 7025.685919] Lustre: DEBUG MARKER: oleg116-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7031.778238] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7031.778899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7031.786340] Lustre: Skipped 1 previous similar message [ 7031.789397] Lustre: Skipped 7 previous similar messages [ 7034.664759] Lustre: server umount lustre-MDT0000 complete [ 7036.274953] LustreError: 380667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1778285091 with bad export cookie 12962769013366907939 [ 7036.278156] LustreError: 381468:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 7036.278651] LustreError: 380667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7036.279571] LustreError: MGC192.168.201.116@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7036.409838] Lustre: server umount lustre-MDT0001 complete [ 7048.301839] Lustre: server umount lustre-OST0000 complete [ 7059.740682] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing set_hostid [ 7063.878812] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing load_modules_local [ 7075.024800] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6720 sec ========== 20:05:30 (1778285130) [ 7075.622179] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:05:30 (1778285130) === [ 7076.822769] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:05:31 (1778285131) === [ 7091.920880] Lustre: DEBUG MARKER: oleg116-server.virtnet: executing unload_modules_local [ 7093.024913] Key type lgssc unregistered [ 7093.157599] LNet: 388604:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7093.160467] LNetError: 388604:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7093.169320] LNet: Removed LNI 192.168.201.116@tcp [ 7093.489779] Key type .llcrypt unregistered [ 7093.491231] Key type ._llcrypt unregistered