[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 432557265 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002337] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004011] kvm-guest: setup PV IPIs [ 0.006700] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007023] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008018] pid_max: default: 32768 minimum: 301 [ 0.010070] LSM: Security Framework initializing [ 0.011058] Yama: becoming mindful. [ 0.012042] SELinux: Initializing. [ 0.013082] *** VALIDATE selinux *** [ 0.020686] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026216] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027190] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028115] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030059] *** VALIDATE tmpfs *** [ 0.031512] *** VALIDATE proc *** [ 0.032252] *** VALIDATE cgroup *** [ 0.033010] *** VALIDATE cgroup2 *** [ 0.034299] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035132] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037038] Spectre V2 : User space: Vulnerable [ 0.038015] Speculative Store Bypass: Vulnerable [ 0.041337] debug: unmapping init [mem 0xffffffffa3659000-0xffffffffa3660fff] [ 0.043193] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044645] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045028] ... version: 2 [ 0.046014] ... bit width: 48 [ 0.047012] ... generic registers: 4 [ 0.048013] ... value mask: 0000ffffffffffff [ 0.049017] ... max period: 00007fffffffffff [ 0.050014] ... fixed-purpose events: 3 [ 0.051010] ... event mask: 000000070000000f [ 0.053268] rcu: Hierarchical SRCU implementation. [ 0.055508] smp: Bringing up secondary CPUs ... [ 0.056611] x86: Booting SMP configuration: [ 0.057026] .... node #0, CPUs: #1 #2 #3 [ 0.060145] smp: Brought up 1 node, 4 CPUs [ 0.062015] smpboot: Max logical packages: 1 [ 0.063018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141020] node 0 deferred pages initialised in 77ms [ 0.143525] devtmpfs: initialized [ 0.144202] x86/mm: Memory block size: 128MB [ 0.146446] gcov: version magic: 0x41383552 [ 0.148049] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.149057] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.150249] pinctrl core: initialized pinctrl subsystem [ 0.151206] [ 0.151607] ************************************************************* [ 0.152008] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.153008] ** ** [ 0.154006] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.155007] ** ** [ 0.156009] ** This means that this kernel is built to expose internal ** [ 0.157008] ** IOMMU data structures, which may compromise security on ** [ 0.158009] ** your system. ** [ 0.159011] ** ** [ 0.160010] ** If you see this message and you are not debugging the ** [ 0.161011] ** kernel, report this immediately to your vendor! ** [ 0.162011] ** ** [ 0.163011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.164013] ************************************************************* [ 0.165672] NET: Registered protocol family 16 [ 0.166428] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.167075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.168075] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.169498] cpuidle: using governor menu [ 0.171109] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.173517] PCI: Using configuration type 1 for base access [ 0.175126] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.183063] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.184046] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.188060] cryptd: max_cpu_qlen set to 1000 [ 0.190328] ACPI: Added _OSI(Module Device) [ 0.191010] ACPI: Added _OSI(Processor Device) [ 0.193012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.193966] ACPI: Added _OSI(Processor Aggregator Device) [ 0.198126] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.203406] ACPI: Interpreter enabled [ 0.205072] ACPI: PM: (supports S0 S3 S4 S5) [ 0.206010] ACPI: Using IOAPIC for interrupt routing [ 0.207101] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.209371] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.217992] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.219031] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.221016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.224087] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.229167] acpiphp: Slot [2] registered [ 0.230100] acpiphp: Slot [5] registered [ 0.232101] acpiphp: Slot [6] registered [ 0.233098] acpiphp: Slot [7] registered [ 0.234127] acpiphp: Slot [8] registered [ 0.236089] acpiphp: Slot [9] registered [ 0.237098] acpiphp: Slot [10] registered [ 0.238071] acpiphp: Slot [3] registered [ 0.239066] acpiphp: Slot [4] registered [ 0.240060] acpiphp: Slot [11] registered [ 0.241064] acpiphp: Slot [12] registered [ 0.242060] acpiphp: Slot [13] registered [ 0.244111] acpiphp: Slot [14] registered [ 0.245055] acpiphp: Slot [15] registered [ 0.245970] acpiphp: Slot [16] registered [ 0.247078] acpiphp: Slot [17] registered [ 0.248066] acpiphp: Slot [18] registered [ 0.249056] acpiphp: Slot [19] registered [ 0.250077] acpiphp: Slot [20] registered [ 0.251074] acpiphp: Slot [21] registered [ 0.253079] acpiphp: Slot [22] registered [ 0.254077] acpiphp: Slot [23] registered [ 0.255060] acpiphp: Slot [24] registered [ 0.257086] acpiphp: Slot [25] registered [ 0.258098] acpiphp: Slot [26] registered [ 0.259098] acpiphp: Slot [27] registered [ 0.261105] acpiphp: Slot [28] registered [ 0.262092] acpiphp: Slot [29] registered [ 0.263092] acpiphp: Slot [30] registered [ 0.265103] acpiphp: Slot [31] registered [ 0.266062] PCI host bridge to bus 0000:00 [ 0.267020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.270031] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.272020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.274028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.277037] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.280040] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.282171] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.283953] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.287303] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.297885] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.302069] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.304012] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.306009] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.308015] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.311089] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.314061] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.316027] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.317766] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.323016] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.333750] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.337010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.341410] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.347012] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.351015] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.368018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.377873] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.383016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.389018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.403016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.413204] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.418017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.423019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.434944] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.443780] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.449012] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.455011] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.471016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.479000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.487012] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.492013] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.508013] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.517346] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.525012] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.532013] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.547012] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.558021] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.560342] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.562381] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.566468] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.569235] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.575045] iommu: Default domain type: Passthrough [ 0.576464] SCSI subsystem initialized [ 0.577119] ACPI: bus type USB registered [ 0.578182] usbcore: registered new interface driver usbfs [ 0.580085] usbcore: registered new interface driver hub [ 0.581083] usbcore: registered new device driver usb [ 0.583184] pps_core: LinuxPPS API ver. 1 registered [ 0.585013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.589077] PTP clock support registered [ 0.591231] EDAC MC: Ver: 3.0.0 [ 0.593133] PCI: Using ACPI for IRQ routing [ 0.594818] NetLabel: Initializing [ 0.596011] NetLabel: domain hash size = 128 [ 0.597010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.599109] NetLabel: unlabeled traffic allowed by default [ 0.602158] vgaarb: loaded [ 0.603305] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.605018] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.612361] clocksource: Switched to clocksource kvm-clock [ 0.705723] VFS: Disk quotas dquot_6.6.0 [ 0.707423] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.710101] *** VALIDATE ramfs *** [ 0.711335] *** VALIDATE hugetlbfs *** [ 0.713504] pnp: PnP ACPI init [ 0.715821] pnp: PnP ACPI: found 6 devices [ 0.729389] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.732530] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.734775] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.736669] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.738794] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.741076] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.743793] NET: Registered protocol family 2 [ 0.746029] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.750405] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.753067] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.757478] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.760833] TCP: Hash tables configured (established 65536 bind 65536) [ 0.763788] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.767023] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.769759] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.772496] NET: Registered protocol family 1 [ 0.774941] RPC: Registered named UNIX socket transport module. [ 0.776735] RPC: Registered udp transport module. [ 0.778540] RPC: Registered tcp transport module. [ 0.780121] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.782396] NET: Registered protocol family 44 [ 0.784299] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.786322] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.788555] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.790660] PCI: CLS 0 bytes, default 64 [ 0.792348] Unpacking initramfs... [ 2.092774] debug: unmapping init [mem 0xffff9a2f3cc54000-0xffff9a2f3ffbffff] [ 2.096204] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.098348] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.100852] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.561165] Initialise system trusted keyrings [ 2.562551] Key type blacklist registered [ 2.564097] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.573047] zbud: loaded [ 2.575963] *** VALIDATE nfs *** [ 2.577249] *** VALIDATE nfs4 *** [ 2.578628] pstore: using deflate compression [ 2.581068] Platform Keyring initialized [ 2.658677] NET: Registered protocol family 38 [ 2.660243] Key type asymmetric registered [ 2.661289] Asymmetric key parser 'x509' registered [ 2.662972] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.666039] io scheduler mq-deadline registered [ 2.667470] io scheduler kyber registered [ 2.669080] io scheduler bfq registered [ 2.670400] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.672601] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.675061] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.678184] ACPI: Power Button [PWRF] [ 2.683027] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 2.688656] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 2.701397] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 2.707535] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 2.723361] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 2.751236] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 2.780351] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 2.784686] Non-volatile memory driver v1.3 [ 2.786074] Linux agpgart interface v0.103 [ 2.813067] virtio_blk virtio1: [vda] 149936 512-byte logical blocks (76.8 MB/73.2 MiB) [ 2.815779] vda: detected capacity change from 0 to 76767232 [ 2.828331] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 2.830819] vdb: detected capacity change from 0 to 1073741824 [ 2.845184] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.848339] vdc: detected capacity change from 0 to 2621440000 [ 2.859279] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 2.862329] vdd: detected capacity change from 0 to 2621440000 [ 2.874662] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.877029] vde: detected capacity change from 0 to 4294967296 [ 2.888039] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 2.890510] vdf: detected capacity change from 0 to 4294967296 [ 2.896523] libphy: Fixed MDIO Bus: probed [ 2.901514] usbcore: registered new interface driver usbserial_generic [ 2.904308] usbserial: USB Serial support registered for generic [ 2.912918] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 2.916692] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 2.918083] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 2.920179] mousedev: PS/2 mouse device common for all mice [ 2.922567] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 2.927754] rtc_cmos 00:05: RTC can wake from S4 [ 2.930395] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 2.932117] rtc_cmos 00:05: registered as rtc0 [ 2.934841] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 2.935305] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 2.941390] intel_pstate: CPU model not supported [ 2.944323] hid: raw HID events driver (C) Jiri Kosina [ 2.946834] usbcore: registered new interface driver usbhid [ 2.949173] usbhid: USB HID core driver [ 2.950494] drop_monitor: Initializing network drop monitor service [ 2.953257] Initializing XFRM netlink socket [ 2.955477] NET: Registered protocol family 10 [ 2.958186] Segment Routing with IPv6 [ 2.959457] NET: Registered protocol family 17 [ 2.961155] mpls_gso: MPLS GSO support [ 2.966109] RAS: Correctable Errors collector initialized. [ 2.967806] AVX version of gcm_enc/dec engaged. [ 2.968916] AES CTR mode by8 optimization enabled [ 3.023256] sched_clock: Marking stable (3023234076, 0)->(3856370592, -833136516) [ 3.026205] registered taskstats version 1 [ 3.027845] Loading compiled-in X.509 certificates [ 3.029463] zswap: loaded using pool lzo/zbud [ 3.048068] Key type big_key registered [ 3.057070] Key type encrypted registered [ 3.058468] ima: No TPM chip found, activating TPM-bypass! [ 3.060155] ima: Allocated hash algorithm: sha1 [ 3.061623] ima: No architecture policies found [ 3.063176] evm: Initialising EVM extended attributes: [ 3.064868] evm: security.selinux [ 3.065837] evm: security.ima [ 3.066903] evm: security.capability [ 3.068184] evm: HMAC attrs: 0x1 [ 3.069808] rtc_cmos 00:05: setting system clock to 2026-09-08 10:21:05 UTC (1788862865) [ 3.075614] debug: unmapping init [mem 0xffffffffa4603000-0xffffffffa47fffff] [ 3.078161] debug: unmapping init [mem 0xffffffffa3382000-0xffffffffa3658fff] [ 3.090113] Write protecting the kernel read-only data: 28672k [ 3.092873] debug: unmapping init [mem 0xffffffffa1a03000-0xffffffffa1bfffff] [ 3.094943] debug: unmapping init [mem 0xffffffffa2314000-0xffffffffa23fffff] [ 3.126404] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.132970] systemd[1]: Detected virtualization kvm. [ 3.134354] systemd[1]: Detected architecture x86-64. [ 3.135777] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.159280] systemd[1]: No hostname configured. [ 3.160592] systemd[1]: Set hostname to . [ 3.162086] random: systemd: uninitialized urandom read (16 bytes read) [ 3.163882] systemd[1]: Initializing machine ID from random generator. [ 3.203820] random: ln: uninitialized urandom read (6 bytes read) [ 3.280645] random: systemd: uninitialized urandom read (16 bytes read) [ 3.283348] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 3.287786] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 3.291624] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 3.826325] device-mapper: uevent: version 1.0.3 [ 3.828714] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 4.487189] virtio_net virtio0 ens2: renamed from eth0 [ 4.513407] random: fast init done [ 4.640648] scsi host0: ata_piix [ 4.649454] scsi host1: ata_piix [ 4.651126] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 4.653652] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.215161] dracut-initqueue[580]: RTNETLINK answers: File exists [ 9.571974] random: crng init done [ 9.573177] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 9.862688] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 10.887260] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.096338] SELinux: Disabled at runtime. [ 11.158971] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.166285] systemd[1]: Detected virtualization kvm. [ 11.167943] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 11.612441] systemd[1]: initrd-switch-root.service: Succeeded. [ 11.616973] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 11.623118] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 11.627925] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 11.632358] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 11.640312] systemd[1]: Starting Journal Service... Starting Journal Service... [ 11.646240] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Initrd Root File System. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. Activating swap /dev/disk/by-label/SWAP... Mounting Huge Pages File System... [ OK ] Reached target Local Encrypted Volumes. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Remount Root and Kernel File Systems... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ 11.775513] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice User and Session Slice. Starting Apply Kernel Variables... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Apply Kernel Variables. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 12.027232] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 12.232255] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 12.261661] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 12.393834] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 12.401481] EDAC sbridge: Ver: 1.1.2 [ 14.082571] Key type dns_resolver registered [ 14.378512] NFS: Registering the id_resolver key type [ 14.380289] Key type id_resolver registered [ 14.381445] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Daily Cleanup of Temporary Directories. Starting Login Service... [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. [ OK ] Started dnf makecache --timer. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Login Service. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg646-server login: [ 38.121250] libcfs: loading out-of-tree module taints kernel. [ 38.140832] Key type ._llcrypt registered [ 38.142439] Key type .llcrypt registered [ 38.188493] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 46.860183] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 47.506573] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 47.518102] alg: No test for adler32 (adler32-zlib) [ 48.566185] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 48.906586] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 50.535386] Key type lgssc registered [ 51.151443] Lustre: Echo OBD driver; http://www.lustre.org/ [ 58.081013] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 73.362881] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 78.136992] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 78.146194] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 79.241642] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 79.254153] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 79.293783] Lustre: lustre-MDT0000: new disk, initializing [ 79.319315] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 79.326569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 80.791636] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 85.976485] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 86.008992] Lustre: 6484:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 86.022307] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 86.025484] Lustre: Skipped 1 previous similar message [ 86.064674] Lustre: lustre-MDT0001: new disk, initializing [ 86.083790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 86.091678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 86.094927] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 87.504781] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 89.824809] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 93.124059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 93.224925] Lustre: lustre-OST0000: new disk, initializing [ 93.228266] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 93.231175] Lustre: 8420:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 93.250540] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 95.313491] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 97.270499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 97.274715] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 97.304613] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 101.153906] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 101.195101] Lustre: lustre-OST0001: new disk, initializing [ 101.196558] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 101.198537] Lustre: 9494:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 101.214423] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 103.099170] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 104.942617] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 104.949274] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 104.964054] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 109.042541] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 112.757266] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 114.656592] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing check_logdir /tmp/testlogs/ [ 116.146824] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing yml_node [ 117.688743] Lustre: DEBUG MARKER: Client: 2.17.58.39 [ 118.464294] Lustre: DEBUG MARKER: MDS: 2.17.58.39 [ 119.260855] Lustre: DEBUG MARKER: OSS: 2.17.58.39 [ 119.782483] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 8 06:23:02 EDT 2026 [ 125.249289] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 125.792848] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 140.767826] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 140.768733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 140.773188] Lustre: Skipped 3 previous similar messages [ 140.775541] Lustre: Skipped 2 previous similar messages [ 143.782848] Lustre: server umount lustre-MDT0000 complete [ 145.105987] LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863007 with bad export cookie 3380170263076649018 [ 145.107934] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 145.110934] LustreError: 6475:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 145.887648] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 145.888023] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 145.892163] Lustre: Skipped 1 previous similar message [ 145.893894] Lustre: Skipped 1 previous similar message [ 151.008706] LustreError: 10072:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 151.008889] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 151.013369] LustreError: 10072:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 151.018421] Lustre: Skipped 2 previous similar messages [ 151.268553] Lustre: server umount lustre-MDT0001 complete [ 158.654860] Lustre: server umount lustre-OST0000 complete [ 166.158448] Lustre: server umount lustre-OST0001 complete [ 168.452975] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 171.683529] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 184.049575] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 184.145544] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 184.159608] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 184.206311] Lustre: lustre-MDT0000: new disk, initializing [ 184.235223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 184.243213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 185.645866] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 189.841160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 189.892439] Lustre: 15035:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 190.005985] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 191.413029] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 195.248143] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 197.177985] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 199.488197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 199.595463] Lustre: lustre-OST0000: new disk, initializing [ 199.598379] Lustre: Skipped 1 previous similar message [ 199.601709] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 199.605814] Lustre: Skipped 2 previous similar messages [ 199.609232] Lustre: 17003:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 199.646457] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 199.649087] Lustre: Skipped 1 previous similar message [ 201.604307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 201.608228] Lustre: Skipped 1 previous similar message [ 201.611220] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 201.629427] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 201.651838] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 205.443678] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 206.888387] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 206.959479] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 208.381304] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 208.453982] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 211.938172] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 211.938678] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 211.941848] LustreError: Skipped 1 previous similar message [ 211.946893] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 211.951149] Lustre: Skipped 1 previous similar message [ 215.189183] Lustre: server umount lustre-OST0000 complete [ 217.058979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 217.061096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 217.066917] Lustre: Skipped 2 previous similar messages [ 222.683444] Lustre: server umount lustre-MDT0000 complete [ 224.006096] LustreError: 15029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863086 with bad export cookie 3380170263076650537 [ 224.009302] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 224.009883] LustreError: 15029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 224.130976] Lustre: server umount lustre-MDT0001 complete [ 225.951472] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:24:48 (1788863088) [ 228.418662] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 228.603662] LustreError: 19264:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 228.631262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 230.113116] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 233.195779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 234.798236] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 237.399050] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 239.400611] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 241.783714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 244.069710] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 246.868183] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 259.387775] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 264.160036] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 264.162405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 264.167252] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 264.169481] Lustre: Skipped 3 previous similar messages [ 268.695955] Lustre: server umount lustre-MDT0000 complete [ 270.056178] LustreError: 21813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863132 with bad export cookie 3380170263076651419 [ 270.059880] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 270.060910] LustreError: 21813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 270.174726] Lustre: server umount lustre-MDT0001 complete [ 281.886258] Lustre: server umount lustre-OST0000 complete [ 289.606981] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 293.461659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 293.621685] LustreError: 23659:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 293.641746] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 293.644469] Lustre: Skipped 2 previous similar messages [ 295.156459] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 298.357481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 299.955705] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 301.020087] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 303.462867] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 305.744682] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 308.978341] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 309.050809] Lustre: lustre-OST0001: new disk, initializing [ 309.052751] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 309.056100] Lustre: 25974:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 310.923534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 310.927129] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 310.975416] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 311.229123] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 315.992948] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 317.387133] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 321.506892] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 321.508747] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 321.513260] Lustre: Skipped 5 previous similar messages [ 321.516351] Lustre: Skipped 5 previous similar messages [ 327.002836] Lustre: server umount lustre-MDT0000 complete [ 328.335196] LustreError: 23640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863190 with bad export cookie 3380170263076652266 [ 328.337699] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 328.339069] LustreError: 23640:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 328.457420] Lustre: server umount lustre-MDT0001 complete [ 339.974481] Lustre: server umount lustre-OST0000 complete [ 351.513844] Lustre: server umount lustre-OST0001 complete [ 356.897766] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 360.449521] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 360.609370] LustreError: 28935:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 360.634458] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 360.637438] Lustre: Skipped 3 previous similar messages [ 362.128664] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 365.239884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 366.953376] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 368.108397] Lustre: 30084:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 370.742627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 371.940656] LustreError: 30437:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 373.276957] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 376.472688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 378.625646] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 381.842318] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 383.047836] Lustre: 31961:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 386.015956] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 386.019042] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 386.024727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 386.027261] Lustre: Skipped 4 previous similar messages [ 390.743509] Lustre: server umount lustre-MDT0000 complete [ 392.012151] LustreError: 28918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863254 with bad export cookie 3380170263076653498 [ 392.015575] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 392.016124] LustreError: 28918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 392.127229] Lustre: server umount lustre-MDT0001 complete [ 402.629389] Lustre: server umount lustre-OST0000 complete [ 414.307895] Lustre: server umount lustre-OST0001 complete [ 417.505633] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 06:28:00 (1788863280) [ 426.904944] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 430.200090] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 430.348467] LustreError: 34756:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 430.355797] LustreError: 34756:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 431.730126] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 434.625569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 436.219345] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 437.243199] Lustre: 35906:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 439.451662] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 441.642420] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 444.698837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 446.956099] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 450.183681] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 451.364262] Lustre: 37779:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 455.138568] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 455.139405] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 455.141605] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 455.141612] Lustre: Skipped 4 previous similar messages [ 455.142775] Lustre: Skipped 5 previous similar messages [ 458.776135] Lustre: server umount lustre-MDT0000 complete [ 459.974188] LustreError: 34739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863322 with bad export cookie 3380170263076654597 [ 459.977210] LustreError: 34739:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 459.977823] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 460.077113] Lustre: server umount lustre-MDT0001 complete [ 471.617188] Lustre: server umount lustre-OST0000 complete [ 483.161159] Lustre: server umount lustre-OST0001 complete [ 486.578893] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 06:29:09 (1788863349) [ 492.955337] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 497.125515] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 497.309793] LustreError: 39915:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 497.320524] LustreError: 39915:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 497.355357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 497.358114] Lustre: Skipped 7 previous similar messages [ 499.112795] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 502.656922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 504.563270] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 505.829143] Lustre: 41063:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 508.691884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 511.292479] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 514.507935] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 516.927470] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 525.791560] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 527.289679] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 527.292181] Lustre: Skipped 1 previous similar message [ 528.352898] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 528.356779] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 528.442473] Lustre: server umount lustre-MDT0000 complete [ 529.742438] LustreError: 39896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863392 with bad export cookie 3380170263076655682 [ 529.746606] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 529.747190] LustreError: 39896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 529.752759] LustreError: 39907:0:(mgc_request.c:614:do_requeue()) failed processing log: -5 [ 531.937779] LustreError: 39912:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 531.943377] LustreError: 39912:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 535.913176] Lustre: server umount lustre-MDT0001 complete [ 538.696986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 539.807146] LustreError: 43787:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 539.810404] LustreError: 43787:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff9a2fa94b1880 x1875758716047488/t0(0) o250->MGC192.168.206.146@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1788863402 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'ptlrpcd_00_02.0' uid:0 gid:0 projid:4294967295 [ 539.816947] LustreError: 43787:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 540.063579] LustreError: 3615:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9a2e826b6680 x1875758716049280/t0(0) o250->MGC192.168.206.146@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 541.493201] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 544.189556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 545.580109] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 547.818278] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 559.718630] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 559.737401] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 11 sec [ 561.561624] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 562.132024] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 564.704531] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 569.564893] Lustre: server umount lustre-MDT0000 complete [ 570.911361] LustreError: 39896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863433 with bad export cookie 3380170263076656732 [ 570.915319] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 570.918765] LustreError: 39896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 571.041324] Lustre: server umount lustre-MDT0001 complete [ 581.632908] Lustre: server umount lustre-OST0000 complete [ 593.047981] Lustre: server umount lustre-OST0001 complete [ 595.754150] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 06:30:58 (1788863458) [ 600.862529] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 604.550589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 604.706300] LustreError: 47554:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 604.714468] LustreError: 47554:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 606.213548] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 609.626908] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 611.582241] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 612.829386] Lustre: 48708:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 612.834014] Lustre: 48708:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 615.432346] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 617.795825] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 620.852041] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 623.172229] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 626.557101] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 628.178395] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 635.360993] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 635.368922] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 635.374868] Lustre: Skipped 16 previous similar messages [ 639.346083] Lustre: server umount lustre-MDT0000 complete [ 640.789373] LustreError: 48313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863503 with bad export cookie 3380170263076657551 [ 640.794520] LustreError: 48313:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 640.796169] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 640.949768] Lustre: server umount lustre-MDT0001 complete [ 652.873733] Lustre: server umount lustre-OST0000 complete [ 663.709542] Lustre: server umount lustre-OST0001 complete [ 666.766724] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 06:32:09 (1788863529) [ 667.655868] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 668.293147] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 06:32:10 (1788863530) [ 672.020598] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 675.705690] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 686.630699] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 686.777032] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 686.833379] Lustre: lustre-MDT0000: new disk, initializing [ 686.920223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 689.525354] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 694.964938] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 698.552180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.695650] Lustre: 54775:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 699.865577] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 699.883026] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 701.413839] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 706.112204] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 707.040390] Lustre: DEBUG MARKER: create test files [ 720.353219] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 720.356533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 720.361263] Lustre: Skipped 12 previous similar messages [ 724.308518] Lustre: server umount lustre-OST0000 complete [ 725.906114] Lustre: server umount lustre-MDT0000 complete [ 730.224043] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 731.328736] Key type lgssc unregistered [ 731.480661] LNet: 56399:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 731.485670] LNetError: 56399:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 731.494429] LNet: Removed LNI 192.168.206.146@tcp [ 731.869132] Key type .llcrypt unregistered [ 731.870490] Key type ._llcrypt unregistered [ 743.971158] Key type ._llcrypt registered [ 743.972779] Key type .llcrypt registered [ 744.023327] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 744.314966] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 744.349713] alg: No test for adler32 (adler32-zlib) [ 745.219193] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 745.319334] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 746.911174] Key type lgssc registered [ 747.321833] Lustre: Echo OBD driver; http://www.lustre.org/ [ 750.466952] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 750.478954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 751.736210] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 753.703902] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 756.827486] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 759.434881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 759.597639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 761.636933] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 761.669432] Lustre: 59112:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 761.672709] Lustre: 59112:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 761.675399] Lustre: 59112:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 761.678549] Lustre: 59112:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 761.681426] Lustre: 59112:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 761.684339] Lustre: 59112:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 762.059759] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 765.251906] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 768.686864] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 770.657773] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 777.184951] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 777.185804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 777.192109] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 778.912734] Lustre: server umount lustre-OST0000 complete [ 780.560574] Lustre: server umount lustre-MDT0000 complete [ 784.421304] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 785.619110] Key type lgssc unregistered [ 785.772602] LNet: 60720:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 785.778364] LNetError: 60720:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 786.790757] LNet: Removed LNI 192.168.206.146@tcp [ 787.155646] Key type .llcrypt unregistered [ 787.157365] Key type ._llcrypt unregistered [ 799.261163] Key type ._llcrypt registered [ 799.262463] Key type .llcrypt registered [ 799.315758] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 799.707330] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 799.724874] alg: No test for adler32 (adler32-zlib) [ 800.596501] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 800.700096] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 802.287189] Key type lgssc registered [ 802.723416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 805.649146] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 805.657544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 806.824237] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 808.406623] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 811.265868] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 813.849557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 813.994137] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 816.277819] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 817.063369] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 817.106306] Lustre: 63519:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 817.110516] Lustre: 63519:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 817.113788] Lustre: 63519:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 817.117243] Lustre: 63519:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 817.120905] Lustre: 63519:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 817.124146] Lustre: 63519:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 819.242710] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 822.635891] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 827.360784] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 827.360942] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 827.371891] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 831.565544] Lustre: server umount lustre-OST0000 complete [ 833.115910] Lustre: server umount lustre-MDT0000 complete [ 837.405098] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 838.590488] Key type lgssc unregistered [ 838.740412] LNet: 65076:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 838.743448] LNetError: 65076:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 838.753450] LNet: Removed LNI 192.168.206.146@tcp [ 839.102541] Key type .llcrypt unregistered [ 839.103835] Key type ._llcrypt unregistered [ 846.438149] Key type ._llcrypt registered [ 846.439806] Key type .llcrypt registered [ 846.484921] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 853.478518] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 853.830337] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 853.856374] alg: No test for adler32 (adler32-zlib) [ 854.724773] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 854.825685] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 856.415279] Key type lgssc registered [ 856.788309] Lustre: Echo OBD driver; http://www.lustre.org/ [ 869.347362] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 869.361684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 870.467404] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 870.487393] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 870.530121] Lustre: lustre-MDT0000: new disk, initializing [ 870.564490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 870.574091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 872.168280] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 877.158504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 877.204582] Lustre: 68198:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 877.232626] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 877.235569] Lustre: Skipped 1 previous similar message [ 877.276029] Lustre: lustre-MDT0001: new disk, initializing [ 877.313776] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 877.326879] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 877.332000] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 878.811529] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 882.865667] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 884.941910] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 887.515300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 887.630429] Lustre: lustre-OST0000: new disk, initializing [ 887.633136] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 887.637671] Lustre: 70163:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 887.670727] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 889.731706] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 889.735841] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 889.779977] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 889.934556] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 893.989035] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 895.611801] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 895.699206] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 897.350332] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 897.438369] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 900.065893] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 900.067230] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 900.070210] LustreError: Skipped 1 previous similar message [ 900.076827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 900.079880] Lustre: Skipped 1 previous similar message [ 904.342836] Lustre: server umount lustre-OST0000 complete [ 908.255846] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 908.259567] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 908.264940] Lustre: Skipped 1 previous similar message [ 908.266651] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 910.304595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 910.305513] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 915.424116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 915.426138] Lustre: Skipped 2 previous similar messages [ 920.031163] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 920.109298] Lustre: server umount lustre-MDT0000 complete [ 921.465629] LustreError: 68189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863783 with bad export cookie 5448461025956304350 [ 921.469546] LustreError: 68189:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 921.470156] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 921.598642] Lustre: server umount lustre-MDT0001 complete [ 924.657184] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 06:36:27 (1788863787) [ 928.806468] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 944.238601] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 948.162239] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 962.925834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 963.037280] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 963.053478] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 963.095290] Lustre: lustre-MDT0000: new disk, initializing [ 963.126147] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 963.134439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 964.823970] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 969.779892] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 969.837650] Lustre: 76182:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 969.865249] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 969.869832] Lustre: Skipped 1 previous similar message [ 969.909572] Lustre: lustre-MDT0001: new disk, initializing [ 969.952613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 969.968679] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 969.973779] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 971.694468] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 976.445430] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 978.833400] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 981.823418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 981.994436] Lustre: lustre-OST0000: new disk, initializing [ 981.997264] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 982.001820] Lustre: 78153:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 982.058340] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 984.450600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 984.457652] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 984.502078] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 985.100740] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 990.010485] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 992.303874] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 992.404820] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 994.300672] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 994.395811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 999.904255] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 999.905952] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 999.909184] LustreError: Skipped 1 previous similar message [ 999.918767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1001.365812] Lustre: server umount lustre-OST0000 complete [ 1005.025451] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1005.032014] Lustre: Skipped 2 previous similar messages [ 1009.254366] Lustre: server umount lustre-MDT0000 complete [ 1010.144446] LustreError: 76193:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1010.153725] LustreError: 76193:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 1010.793784] LustreError: 76173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863873 with bad export cookie 5448461025956305218 [ 1010.795435] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1010.798836] LustreError: 76173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1010.935949] Lustre: server umount lustre-MDT0001 complete [ 1014.354907] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 06:37:56 (1788863876) [ 1017.180745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1017.400564] LustreError: 80559:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1017.437271] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1019.178343] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1022.433514] LustreError: 80578:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1022.817271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1024.709859] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1027.911423] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1030.268328] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1033.202242] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1033.251900] LustreError: 82378:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1035.984817] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1039.089612] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1040.559518] Lustre: 83225:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting ff2f21d8-d5ee-4d82-a551-75121172507a at adminstrative request [ 1041.893351] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1041.900762] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1052.134197] Lustre: MGS: Client ff2f21d8-d5ee-4d82-a551-75121172507a (at 0@lo) reconnecting [ 1052.137440] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x4b9cd26f98af9cb4 to 0x4b9cd26f98af9f54 [ 1052.142946] Lustre: MGC192.168.206.146@tcp: Connection restored to 0@lo (at 0@lo) [ 1073.625710] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1073.719854] Lustre: lustre-OST0001: new disk, initializing [ 1073.722363] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1073.727785] Lustre: 83619:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1073.767134] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1073.772656] Lustre: Skipped 2 previous similar messages [ 1076.233726] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1080.416697] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1081.539941] Lustre: server umount lustre-OST0001 complete [ 1082.888195] LustreError: 82379:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1084.895853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1084.900561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1084.908790] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1084.912488] Lustre: Skipped 3 previous similar messages [ 1089.174285] Lustre: server umount lustre-OST0000 complete [ 1096.745302] Lustre: server umount lustre-MDT0000 complete [ 1098.208650] LustreError: 80578:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1098.227085] LustreError: 80578:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [ 1098.442674] LustreError: 80539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788863960 with bad export cookie 5448461025956306772 [ 1098.447601] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1098.450713] LustreError: 80539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1098.660081] Lustre: server umount lustre-MDT0001 complete [ 1102.951722] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 06:39:25 (1788863965) [ 1103.666821] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1104.424908] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 06:39:26 (1788863966) [ 1105.058463] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1105.766722] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 06:39:28 (1788863968) [ 1106.470629] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1107.185595] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 06:39:29 (1788863969) [ 1107.791195] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1108.493452] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 06:39:31 (1788863971) [ 1109.194666] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1111.768881] loop: module loaded [ 1111.776286] loop0: detected capacity change from 0 to 4294967296 [ 1111.801205] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1112.504035] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1113.897913] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1115.623936] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1126.697586] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1126.715576] LustreError: 87732:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1126.718259] LustreError: 87732:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1126.721652] LustreError: 87732:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1126.725597] LustreError: 87732:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1134.298665] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 1138.220492] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1152.845863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1152.955604] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1152.972966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1153.011889] Lustre: lustre-MDT0000: new disk, initializing [ 1153.044144] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1153.053512] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1154.923282] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1160.221316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1160.271243] Lustre: 90593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1160.402154] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1162.218496] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1166.912807] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1169.425827] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1172.259155] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1172.369278] Lustre: 92557:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1174.687786] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1178.120505] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1178.162749] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1178.896719] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1180.631208] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1180.741948] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1182.462964] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1182.562711] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1188.320600] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1188.323482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1188.324721] LustreError: Skipped 2 previous similar messages [ 1188.330669] Lustre: Skipped 3 previous similar messages [ 1188.333979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1188.337997] Lustre: Skipped 5 previous similar messages [ 1189.536868] Lustre: server umount lustre-OST0000 complete [ 1197.280962] Lustre: server umount lustre-MDT0000 complete [ 1198.560121] LustreError: 90604:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1198.567467] LustreError: 90604:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 1198.938445] LustreError: 90585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864061 with bad export cookie 5448461025956307374 [ 1198.942434] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1198.943551] LustreError: 90585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1199.074746] Lustre: server umount lustre-MDT0001 complete [ 1202.687358] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 06:41:05 (1788864065) [ 1205.611149] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1207.670905] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1211.148767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1213.012350] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1215.806738] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1217.869068] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1220.448338] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1223.215608] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1226.385895] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1228.251947] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1231.414140] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 3 sec [ 1233.152705] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1233.241955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1235.984399] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1236.061817] Lustre: lustre-OST0001: new disk, initializing [ 1236.063745] Lustre: Skipped 2 previous similar messages [ 1236.065682] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1236.069278] Lustre: Skipped 3 previous similar messages [ 1236.072789] Lustre: 98356:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1238.353101] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1242.124065] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1242.127853] Lustre: Skipped 2 previous similar messages [ 1242.130939] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1242.173951] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1242.378471] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1244.086045] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1244.172477] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1245.742601] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1245.840193] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1256.400623] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1263.455694] Lustre: Failing over lustre-MDT0000 [ 1263.847300] Lustre: server umount lustre-MDT0000 complete [ 1267.680951] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1267.683292] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1267.683952] LustreError: Skipped 1 previous similar message [ 1267.684641] LustreError: 94964:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1267.684647] LustreError: 94964:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 1267.690316] Lustre: Skipped 3 previous similar messages [ 1279.563309] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1279.565023] LDISKFS-fs (dm-0): recovery complete [ 1279.568780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1279.622242] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1279.762479] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1281.410664] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1283.036468] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1283.036604] Lustre: 96300:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1285.092334] Lustre: 94968:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1285.096648] Lustre: 94968:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 1285.101877] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1285.124285] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1285.439121] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1285.761262] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1285.765864] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 1286.079116] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1286.083058] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 1286.401380] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1286.406550] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 8 previous similar messages [ 1287.033260] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1287.036817] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 3 previous similar messages [ 1287.351085] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1287.355639] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 3 previous similar messages [ 1288.607828] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1288.611826] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 1289.240129] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1289.245278] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 1289.559115] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1289.562480] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 1292.697812] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1292.702947] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 1293.329115] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1293.332729] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 1293.647111] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1293.651925] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 1300.833399] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1300.837457] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 25 previous similar messages [ 1301.465127] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1301.469341] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 1301.783154] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1301.786701] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 1317.097409] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1317.101786] Lustre: 101174:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 51 previous similar messages [ 1317.729341] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1317.733831] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 51 previous similar messages [ 1318.047110] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1318.051062] LustreError: 101174:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 51 previous similar messages [ 1343.071331] Lustre: 101174:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1343.075369] Lustre: 101174:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1343.078865] Lustre: 101174:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 4ee93e48-ff69-4aec-9906-ed48b2971942@192.168.206.46@tcp [ 1343.084672] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1343.087493] Lustre: 101174:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@0000000068631fc3 x1875759554695680/t0(8589934782) o35->4ee93e48-ff69-4aec-9906-ed48b2971942@192.168.206.46@tcp:436/0 lens 392/0 e 0 to 0 dl 1788864216 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1343.098955] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1343.103741] Lustre: lustre-MDT0000: Denying connection for new client 4ee93e48-ff69-4aec-9906-ed48b2971942 (at 192.168.206.46@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1343.107806] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1343.124248] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1343.127424] Lustre: Skipped 2 previous similar messages [ 1343.131279] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1343.150896] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:49 to 0x2c0000401:545) [ 1343.150947] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 1351.654884] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1351.656217] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1351.663220] Lustre: Skipped 2 previous similar messages [ 1351.665901] Lustre: Skipped 3 previous similar messages [ 1357.119745] Lustre: server umount lustre-OST0000 complete [ 1358.817291] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1361.889513] LustreError: 96938:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1361.895927] LustreError: 96938:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 18 previous similar messages [ 1364.629861] Lustre: server umount lustre-OST0001 complete [ 1380.319182] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1380.408154] Lustre: server umount lustre-MDT0000 complete [ 1381.897165] LustreError: 96810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864244 with bad export cookie 5448461025956358558 [ 1381.900485] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1381.903067] LustreError: 96810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1382.036154] Lustre: server umount lustre-MDT0001 complete [ 1385.400396] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 06:44:08 (1788864248) [ 1388.115932] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1388.120219] Lustre: *** cfs_fail_loc=197, val=0*** [ 1403.615662] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1408.735425] LustreError: 103151:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.146@tcp: failed processing log, type 4: rc = -110 [ 1413.863883] Lustre: *** cfs_fail_loc=197, val=0*** [ 1413.865965] Lustre: Skipped 65 previous similar messages [ 1434.335581] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1434.338353] Lustre: Skipped 7 previous similar messages [ 1436.559063] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1439.321969] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1440.147690] Lustre: server umount lustre-OST0000 complete [ 1443.281739] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 06:45:05 (1788864305) [ 1457.517811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1459.230770] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1463.357742] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1465.562494] LustreError: 104800:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1465.567432] LustreError: 104800:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1465.571749] LustreError: 104800:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -5 [ 1465.714580] Lustre: server umount lustre-MDT0000 complete [ 1478.705781] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 1482.477473] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1496.983638] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1497.089593] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1497.106756] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1497.147142] Lustre: lustre-MDT0000: new disk, initializing [ 1497.197091] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1498.982733] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1504.117929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1504.167261] Lustre: 108965:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1504.203929] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 1504.206572] Lustre: Skipped 12 previous similar messages [ 1504.296154] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1505.962984] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1510.350468] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1512.602978] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1515.248298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1515.368376] Lustre: 110932:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1517.805315] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1518.513850] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1522.137180] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1524.022339] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1524.127852] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1525.853265] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1525.956121] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1528.803086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1528.805147] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1528.809575] Lustre: Skipped 4 previous similar messages [ 1528.817056] LustreError: Skipped 1 previous similar message [ 1532.963294] Lustre: server umount lustre-OST0000 complete [ 1540.640815] Lustre: server umount lustre-MDT0000 complete [ 1542.143206] LustreError: 108956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864404 with bad export cookie 5448461025956369268 [ 1542.144909] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1542.149404] LustreError: 108956:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1542.291947] Lustre: server umount lustre-MDT0001 complete [ 1545.806774] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 06:46:48 (1788864408) [ 1549.260389] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 1550.495456] Key type lgssc unregistered [ 1550.657584] LNet: 113415:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1550.661804] LNetError: 113415:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1550.671375] LNet: Removed LNI 192.168.206.146@tcp [ 1551.038586] Key type .llcrypt unregistered [ 1551.039959] Key type ._llcrypt unregistered [ 1557.712881] Key type ._llcrypt registered [ 1557.714338] Key type .llcrypt registered [ 1557.757566] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 1564.792950] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1565.157552] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1565.179362] alg: No test for adler32 (adler32-zlib) [ 1566.046508] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 1566.140913] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 1567.735198] Key type lgssc registered [ 1568.187524] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1584.373483] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1584.383226] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1585.492102] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1585.507560] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1585.547156] Lustre: lustre-MDT0000: new disk, initializing [ 1585.581702] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1585.593920] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1587.165924] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1592.118558] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1592.239457] Lustre: lustre-OST0000: new disk, initializing [ 1592.242271] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1592.244856] Lustre: Skipped 1 previous similar message [ 1592.247125] Lustre: 117554:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1592.278443] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1594.634256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1594.638592] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1594.652297] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1594.707928] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1599.746358] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1599.835211] Lustre: lustre-OST0001: new disk, initializing [ 1599.838202] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1599.844321] Lustre: 118417:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1599.877549] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1602.366484] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1603.548533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 1603.554437] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 1603.567112] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 1608.674945] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1608.676264] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1608.686485] Lustre: Skipped 1 previous similar message [ 1612.939184] Lustre: server umount lustre-MDT0000 complete [ 1615.715915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 1617.621822] LustreError: 116743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864480 with bad export cookie 13820048645945548089 [ 1617.623844] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1617.626512] LustreError: 116743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1617.666862] Lustre: server umount lustre-OST0000 complete [ 1619.232449] Lustre: server umount lustre-OST0001 complete [ 1622.522215] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 06:48:05 (1788864485) [ 1630.426210] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 06:48:13 (1788864493) [ 1633.950900] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1642.639709] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 06:48:25 (1788864505) [ 1656.993757] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 1660.627585] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1674.762869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1674.858981] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1674.872375] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1674.915269] Lustre: lustre-MDT0000: new disk, initializing [ 1674.952300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1674.963677] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1676.545937] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1681.260549] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1681.322113] Lustre: 126154:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1681.352169] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1681.355041] Lustre: Skipped 1 previous similar message [ 1681.394844] Lustre: lustre-MDT0001: new disk, initializing [ 1681.440779] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1681.456460] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1681.462407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1683.152534] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1687.495129] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1689.810259] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1692.932889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1693.065968] Lustre: lustre-OST0000: new disk, initializing [ 1693.069888] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1693.074952] Lustre: 128122:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1693.122363] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1696.130996] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1698.833095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1698.840374] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1698.904062] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1701.457548] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1703.564309] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1703.665249] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1705.615941] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1705.704908] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1709.027343] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1709.027451] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1709.027996] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1709.028018] Lustre: Skipped 1 previous similar message [ 1709.030214] LustreError: Skipped 1 previous similar message [ 1709.035523] Lustre: Skipped 1 previous similar message [ 1712.598822] Lustre: server umount lustre-OST0000 complete [ 1714.144517] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1714.145288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1714.149772] Lustre: Skipped 1 previous similar message [ 1714.152209] Lustre: Skipped 1 previous similar message [ 1719.264686] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1719.268950] Lustre: Skipped 2 previous similar messages [ 1720.222910] Lustre: server umount lustre-MDT0000 complete [ 1721.708394] LustreError: 126950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864584 with bad export cookie 13820048645945549300 [ 1721.710807] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1721.713658] LustreError: 126950:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1721.836742] Lustre: server umount lustre-MDT0001 complete [ 1725.190661] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 06:49:47 (1788864587) [ 1727.792636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1728.010077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.597399] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1732.905122] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1734.724203] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1737.641578] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1740.051531] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1742.792444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1745.301629] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1748.249605] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1755.577789] Lustre: *** cfs_fail_loc=159, val=0*** [ 1770.994082] Lustre: lustre-MDT0000: Client 1e725a68-0cc5-4066-824e-b315735a8d6b (at 192.168.206.46@tcp) reconnecting [ 1771.592659] Lustre: *** cfs_fail_loc=159, val=0*** [ 1771.594525] Lustre: Skipped 5 previous similar messages [ 1787.870543] Lustre: lustre-MDT0000: Client 1e725a68-0cc5-4066-824e-b315735a8d6b (at 192.168.206.46@tcp) reconnecting [ 1789.411866] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1789.414037] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1789.422073] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1789.425572] Lustre: Skipped 1 previous similar message [ 1795.430598] Lustre: server umount lustre-OST0000 complete [ 1799.648909] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1799.649263] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1799.649770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1799.649775] Lustre: Skipped 2 previous similar messages [ 1799.660413] Lustre: Skipped 2 previous similar messages [ 1803.171466] Lustre: server umount lustre-MDT0000 complete [ 1804.756531] LustreError: 130512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864667 with bad export cookie 13820048645945550168 [ 1804.759182] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1804.763096] LustreError: 130512:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1804.896410] Lustre: server umount lustre-MDT0001 complete [ 1809.462585] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 1810.785307] Key type lgssc unregistered [ 1810.943515] LNet: 134300:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1810.946376] LNetError: 134300:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1810.960301] LNet: Removed LNI 192.168.206.146@tcp [ 1811.321411] Key type .llcrypt unregistered [ 1811.322994] Key type ._llcrypt unregistered [ 1822.208665] Key type ._llcrypt registered [ 1822.210333] Key type .llcrypt registered [ 1822.260279] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1822.580932] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1822.590319] alg: No test for adler32 (adler32-zlib) [ 1823.462524] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 1823.566952] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 1825.175198] Key type lgssc registered [ 1825.631266] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1828.670977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1828.678941] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1829.870927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1831.422793] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1834.574452] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1834.728188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1836.295554] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1839.133861] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1841.545294] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1844.812671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1844.975165] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1847.744052] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1850.680357] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1855.465603] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 1860.612570] Lustre: *** cfs_fail_loc=159, val=0*** [ 1875.953493] Lustre: lustre-MDT0000: Client ce11ee31-9594-4a31-b53e-c078cf066316 (at 192.168.206.46@tcp) reconnecting [ 1876.793541] Lustre: *** cfs_fail_loc=159, val=0*** [ 1876.795816] Lustre: Skipped 2 previous similar messages [ 1892.336383] Lustre: lustre-MDT0001: Client ce11ee31-9594-4a31-b53e-c078cf066316 (at 192.168.206.46@tcp) reconnecting [ 1892.893779] Lustre: *** cfs_fail_loc=159, val=0*** [ 1909.236556] Lustre: lustre-MDT0001: Client ce11ee31-9594-4a31-b53e-c078cf066316 (at 192.168.206.46@tcp) reconnecting [ 1910.590360] Lustre: 138721:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1911.648253] Lustre: *** cfs_fail_loc=159, val=0*** [ 1911.650386] Lustre: Skipped 4 previous similar messages [ 1927.155615] Lustre: lustre-MDT0000: Client df1ff5e6-7ccb-4bc3-b6f0-e341680577c3 (at 192.168.206.46@tcp) reconnecting [ 1927.859281] Lustre: *** cfs_fail_loc=159, val=0*** [ 1927.861165] Lustre: Skipped 14 previous similar messages [ 1943.026839] Lustre: lustre-MDT0000: Client df1ff5e6-7ccb-4bc3-b6f0-e341680577c3 (at 192.168.206.46@tcp) reconnecting [ 1943.643867] Lustre: 139012:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 1947.619674] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1947.621856] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1947.623765] LustreError: Skipped 1 previous similar message [ 1947.624079] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1947.632021] Lustre: Skipped 1 previous similar message [ 1951.317091] Lustre: server umount lustre-OST0000 complete [ 1952.737365] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1952.737618] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1952.744728] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1952.748715] Lustre: Skipped 2 previous similar messages [ 1957.856403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1957.859601] Lustre: Skipped 1 previous similar message [ 1958.878824] Lustre: server umount lustre-MDT0000 complete [ 1960.247271] LustreError: 138501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864822 with bad export cookie 7284823051568107321 [ 1960.250474] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1960.252282] LustreError: 138501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1960.379431] Lustre: server umount lustre-MDT0001 complete [ 1964.297219] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 1965.435255] Key type lgssc unregistered [ 1965.578717] LNet: 140062:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1965.581896] LNetError: 140062:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1965.594519] LNet: Removed LNI 192.168.206.146@tcp [ 1965.953049] Key type .llcrypt unregistered [ 1965.954608] Key type ._llcrypt unregistered [ 1975.714035] Key type ._llcrypt registered [ 1975.715368] Key type .llcrypt registered [ 1975.756236] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 1976.089519] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1976.127763] alg: No test for adler32 (adler32-zlib) [ 1977.008602] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 1977.098545] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 1978.687217] Key type lgssc registered [ 1979.093135] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1981.883982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1981.891500] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1983.054584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1984.527781] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1987.595261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1987.743776] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1989.281542] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 1991.850566] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1993.831419] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1996.483928] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1996.656975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1997.670932] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 1998.997558] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2001.754394] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2007.918338] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2014.532722] Lustre: 144051:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2015.271781] Lustre: 144099:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2015.789607] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2016.981484] Lustre: 144244:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2017.277139] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2017.280366] Lustre: Skipped 1 previous similar message [ 2017.595995] Lustre: 144342:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2023.396385] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2023.397838] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2023.402258] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2023.410139] Lustre: Skipped 1 previous similar message [ 2024.729859] Lustre: server umount lustre-OST0000 complete [ 2028.514093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2028.519528] Lustre: Skipped 1 previous similar message [ 2028.522617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2028.525892] Lustre: Skipped 1 previous similar message [ 2032.359418] Lustre: server umount lustre-MDT0000 complete [ 2033.632742] LustreError: 141449:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2033.640058] LustreError: 141449:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2033.798429] LustreError: 141429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864896 with bad export cookie 3487458319919714921 [ 2033.801963] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2033.806700] LustreError: 141429:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2033.926800] Lustre: server umount lustre-MDT0001 complete [ 2038.270482] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 2039.467033] Key type lgssc unregistered [ 2039.619447] LNet: 145400:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2039.623300] LNetError: 145400:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2039.634631] LNet: Removed LNI 192.168.206.146@tcp [ 2039.983186] Key type .llcrypt unregistered [ 2039.985276] Key type ._llcrypt unregistered [ 2049.940601] Key type ._llcrypt registered [ 2049.942311] Key type .llcrypt registered [ 2049.985902] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2050.341107] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2050.411851] alg: No test for adler32 (adler32-zlib) [ 2051.309991] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 2051.444444] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 2053.047197] Key type lgssc registered [ 2053.495248] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2056.601085] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2056.610900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2057.769572] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2059.259906] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2062.355329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2062.494995] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2063.896836] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2066.537119] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2068.443439] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2070.918310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2071.075401] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2072.101609] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2073.089095] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2075.858278] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2083.365717] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2089.626789] Lustre: *** cfs_fail_loc=159, val=0*** [ 2105.839569] Lustre: lustre-MDT0000: Client 2a10e714-6f75-46bf-af70-4866d6d8487a (at 192.168.206.46@tcp) reconnecting [ 2123.231808] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2123.235180] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2123.239568] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2124.258629] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2124.261880] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2124.262043] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2126.868306] Lustre: server umount lustre-OST0000 complete [ 2129.377625] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2129.378520] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2129.388193] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2129.394377] Lustre: Skipped 2 previous similar messages [ 2134.434915] Lustre: server umount lustre-MDT0000 complete [ 2134.496519] LustreError: 146789:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2134.506369] LustreError: 146789:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2135.996908] LustreError: 149418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788864998 with bad export cookie 12856914551637376074 [ 2136.000409] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2136.004117] LustreError: 149418:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2136.133446] Lustre: server umount lustre-MDT0001 complete [ 2140.483786] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 2141.627723] Key type lgssc unregistered [ 2141.762483] LNet: 150470:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2141.765573] LNetError: 150470:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2141.779400] LNet: Removed LNI 192.168.206.146@tcp [ 2142.139216] Key type .llcrypt unregistered [ 2142.140873] Key type ._llcrypt unregistered [ 2152.495576] Key type ._llcrypt registered [ 2152.497085] Key type .llcrypt registered [ 2152.544805] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2152.967808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2152.973872] alg: No test for adler32 (adler32-zlib) [ 2153.834163] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 2153.932992] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 2155.519189] Key type lgssc registered [ 2155.911959] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2158.705246] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2158.714038] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2159.881214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2161.443462] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2164.973191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2165.150617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2166.656409] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2169.418948] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2171.558232] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2174.107208] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2174.230635] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2175.274107] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2176.370331] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2178.971376] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2185.511759] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2202.169753] Lustre: lustre-OST0000: evicting a1e029af-6c75-4c1b-8393-9720b488b992 (at 192.168.206.46@tcp) by administrative request [ 2202.174379] Lustre: Skipped 1 previous similar message [ 2226.659065] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2226.659250] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2226.667519] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2226.670084] LustreError: Skipped 1 previous similar message [ 2226.673396] Lustre: Skipped 1 previous similar message [ 2231.779282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2231.781775] Lustre: Skipped 1 previous similar message [ 2232.658561] Lustre: server umount lustre-OST0000 complete [ 2236.896915] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2236.897529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2236.901939] Lustre: Skipped 2 previous similar messages [ 2236.906678] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2240.159772] Lustre: server umount lustre-MDT0000 complete [ 2241.578277] LustreError: 151839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788865104 with bad export cookie 284392686322276970 [ 2241.580806] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2241.582260] LustreError: 151839:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2241.719329] Lustre: server umount lustre-MDT0001 complete [ 2245.593794] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 2246.738985] Key type lgssc unregistered [ 2246.873501] LNet: 155938:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2246.876353] LNetError: 155938:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2246.885301] LNet: Removed LNI 192.168.206.146@tcp [ 2247.225681] Key type .llcrypt unregistered [ 2247.227568] Key type ._llcrypt unregistered [ 2272.731485] Key type ._llcrypt registered [ 2272.732596] Key type .llcrypt registered [ 2272.771507] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2273.106290] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2273.152150] alg: No test for adler32 (adler32-zlib) [ 2274.024300] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 2274.118091] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 2275.703243] Key type lgssc registered [ 2276.065687] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2278.618044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2278.623323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2279.764721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2281.202137] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2284.357791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2284.532330] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2286.108587] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2288.831617] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2290.836036] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2293.304552] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2293.453954] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2294.502282] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2295.618613] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2298.681309] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2304.743441] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2310.115169] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2310.117112] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2310.119313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2310.131634] Lustre: Skipped 1 previous similar message [ 2314.455132] Lustre: server umount lustre-OST0000 complete [ 2320.352735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2320.353882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2320.358750] Lustre: Skipped 1 previous similar message [ 2320.361625] Lustre: Skipped 1 previous similar message [ 2320.364957] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2320.370772] LustreError: Skipped 1 previous similar message [ 2322.268864] Lustre: server umount lustre-MDT0000 complete [ 2323.867791] LustreError: 158374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788865186 with bad export cookie 10755352200135583954 [ 2323.871652] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2323.874248] LustreError: 158374:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2324.013697] Lustre: server umount lustre-MDT0001 complete [ 2328.167714] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 2329.416165] Key type lgssc unregistered [ 2329.572868] LNet: 161932:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2329.578635] LNetError: 161932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2329.588747] LNet: Removed LNI 192.168.206.146@tcp [ 2329.987261] Key type .llcrypt unregistered [ 2329.988493] Key type ._llcrypt unregistered [ 2340.275035] Key type ._llcrypt registered [ 2340.276601] Key type .llcrypt registered [ 2340.313996] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2340.640819] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2340.660079] alg: No test for adler32 (adler32-zlib) [ 2341.558683] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 2341.658515] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 2343.247182] Key type lgssc registered [ 2343.703897] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2349.695415] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 07:00:12 (1788865212) [ 2355.456570] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 2359.145231] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2372.705809] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2372.712956] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2373.808844] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2373.823851] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2373.866114] Lustre: lustre-MDT0000: new disk, initializing [ 2373.907138] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2373.916419] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2375.438115] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2379.931483] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2379.977632] Lustre: 166160:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2380.003408] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2380.006155] Lustre: Skipped 1 previous similar message [ 2380.043151] Lustre: lustre-MDT0001: new disk, initializing [ 2380.075597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2380.087329] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2380.090382] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2381.563409] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2385.637129] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2387.811019] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2390.132576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2390.237143] Lustre: lustre-OST0000: new disk, initializing [ 2390.239273] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2390.242637] Lustre: 168128:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2390.273888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2392.253631] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2392.256883] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2392.293825] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2392.320752] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2396.531421] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2402.786557] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2402.786786] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2402.789429] LustreError: Skipped 1 previous similar message [ 2402.790263] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2402.795189] Lustre: Skipped 1 previous similar message [ 2402.800044] Lustre: Skipped 1 previous similar message [ 2407.129362] Lustre: server umount lustre-OST0000 complete [ 2410.976784] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2410.980358] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2410.985356] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2413.024520] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2413.025620] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2414.619216] Lustre: server umount lustre-MDT0000 complete [ 2415.969541] LustreError: 169008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788865278 with bad export cookie 7251129690435944266 [ 2415.972745] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2415.972981] LustreError: 169008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2416.109663] Lustre: server umount lustre-MDT0001 complete [ 2420.045803] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 2421.183947] Key type lgssc unregistered [ 2421.326433] LNet: 170045:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2421.330403] LNetError: 170045:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2421.339799] LNet: Removed LNI 192.168.206.146@tcp [ 2421.721483] Key type .llcrypt unregistered [ 2421.723388] Key type ._llcrypt unregistered [ 2431.613411] Key type ._llcrypt registered [ 2431.614964] Key type .llcrypt registered [ 2431.658222] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 2432.000952] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2432.028707] alg: No test for adler32 (adler32-zlib) [ 2432.902897] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 2433.000151] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 2434.599168] Key type lgssc registered [ 2435.014464] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2437.807578] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2437.814445] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2438.959680] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2440.456912] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2443.694733] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2443.839835] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2445.441800] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2448.330805] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2450.500485] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2452.932195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2453.061536] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2455.264413] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 2457.969206] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2465.382700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 2470.987679] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2470.995575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.003903] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2471.008516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.014572] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2471.438539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.837993] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2471.841637] Lustre: Skipped 1 previous similar message [ 2471.850757] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2471.855369] Lustre: Skipped 1 previous similar message [ 2471.857693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2471.861526] Lustre: Skipped 1 previous similar message [ 2471.863991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2471.869726] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2471.872934] Lustre: Skipped 1 previous similar message [ 2472.240577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2472.620872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.046829] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2473.050318] Lustre: Skipped 2 previous similar messages [ 2473.057302] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2473.065455] Lustre: Skipped 2 previous similar messages [ 2473.069252] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2473.074525] Lustre: Skipped 2 previous similar messages [ 2473.076360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.082056] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2473.085041] Lustre: Skipped 2 previous similar messages [ 2473.443150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2473.846127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.259681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2474.651498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.057610] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2475.061089] Lustre: Skipped 4 previous similar messages [ 2475.067587] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2475.071556] Lustre: Skipped 4 previous similar messages [ 2475.075591] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2475.080270] Lustre: Skipped 4 previous similar messages [ 2475.082487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.087629] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2475.090286] Lustre: Skipped 4 previous similar messages [ 2475.447287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2475.802891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.171683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.519165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2476.868473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.227111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.605348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2477.964788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.319844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2478.719083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.078239] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2479.082046] Lustre: Skipped 10 previous similar messages [ 2479.089377] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2479.095381] Lustre: Skipped 10 previous similar messages [ 2479.097568] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2479.100594] Lustre: Skipped 10 previous similar messages [ 2479.102746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.107332] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2479.110018] Lustre: Skipped 10 previous similar messages [ 2479.468874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2479.836847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.216288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.626244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2480.993052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.355884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2481.739514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.107825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.523755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2482.902032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.294344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2483.666273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.086970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.485646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2484.882112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.284674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.642523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2485.997820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.361162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2486.742841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.092339] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2487.095574] Lustre: Skipped 20 previous similar messages [ 2487.102750] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2487.107840] Lustre: Skipped 20 previous similar messages [ 2487.110677] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2487.113578] Lustre: Skipped 20 previous similar messages [ 2487.116732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.121525] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2487.124694] Lustre: Skipped 20 previous similar messages [ 2487.461705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2487.814720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.151830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.512626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2488.877228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.241576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.576881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2489.933172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.277470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.615962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2490.982110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.327356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2491.680285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.029287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.357745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2492.692371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.035989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.392331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2493.760924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.116867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.471544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2494.813193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.177656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.571791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2495.941442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.310431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2496.674978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.031368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.408858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2497.767996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.136757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.514150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2498.886889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.256533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.618414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2499.986868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.353370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2500.738655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.115234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.474196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2501.860332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.200660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.571612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2502.952596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.343114] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2503.347259] Lustre: Skipped 44 previous similar messages [ 2503.356728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2503.363596] Lustre: Skipped 44 previous similar messages [ 2503.367286] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2503.370207] Lustre: Skipped 44 previous similar messages [ 2503.373659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2503.379119] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2503.382423] Lustre: Skipped 44 previous similar messages [ 2503.741079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.093255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.458721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2504.813711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.177577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.546108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2505.908115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.287327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.628704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2506.985919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.378644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2507.753172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.158336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.548426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2508.927862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.341543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2509.751228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.146305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.525158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2510.890392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.288090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2511.684895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.074523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.426090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2512.783161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.168205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.544719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2513.920799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.296477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2514.703562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.102385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.505668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2515.959294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.351607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2516.761178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.198290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.607046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2517.999701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.368619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2518.718840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.079902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.455520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2519.834610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.193501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.557870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2520.913648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.307337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2521.700213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.094734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.523229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2522.953368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.336784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2523.748158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.191302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2524.615641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.004260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.400098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2525.811657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.200293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.625715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2526.999632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.378516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2527.797254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.161362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.546312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2528.937760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.297258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2529.711786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.103259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.472101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2530.825554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.182860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.569069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2531.944579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.334641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2532.745992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.182443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.561370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2533.955576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.377579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2534.779058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.192554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.602639] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2535.605516] Lustre: Skipped 82 previous similar messages [ 2535.612353] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2535.617971] Lustre: Skipped 82 previous similar messages [ 2535.620731] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2535.623801] Lustre: Skipped 82 previous similar messages [ 2535.625322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2535.629318] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2535.632618] Lustre: Skipped 82 previous similar messages [ 2536.076823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.518221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2536.932912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.350258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2537.773380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.186051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.579667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2538.976072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.368409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2539.768656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.130606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.488864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2540.869415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.275549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2541.635937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.013613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.380164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2542.779268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.161625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.540638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2543.892034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.253856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2544.610629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.005922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.376936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2545.723728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.097900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.458213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2546.818092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.202179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.582227] LustreError: 171450:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2547.586461] LustreError: 171450:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2547.587899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2547.963394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.336083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2548.721268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.077475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.495380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2549.870746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.234215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.576220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2550.941150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.282372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.630462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2551.991331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.379982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2552.768927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.193275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2553.626078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.047208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.480181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2554.900501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.299441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2555.737113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.128639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.537351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2556.916555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.307608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2557.701139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.111795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.557483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2558.956241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.370398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2559.809556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.245584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2560.655451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.052214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.474121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2561.885069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.283159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2562.687909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.139663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.568697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2563.990723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.440191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2564.895307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.319583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2565.778168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.180568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2566.602145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.028904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.416161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2567.827554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.239660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2568.638338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.000872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.380793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2569.733291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.084198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.446402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2570.813108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.179183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.554313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2571.888000] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.241235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.562313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2572.934661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.260272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.584596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2573.921217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.274097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.619470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2574.978058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.334802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2575.690896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.031866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.367705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2576.710534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.043287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.403507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2577.746433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.096042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.442776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.811640] LustreError: 171450:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2578.817701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2578.818977] LustreError: 171450:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2579.167655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.512407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2579.836073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.199455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.530762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2580.862784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.199567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.558737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2581.883052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.193071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.485505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2582.792946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.127565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.444202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2583.751323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.086279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.404983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2584.731343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.052802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.380961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2585.701415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.026736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.364385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2586.720493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.052786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.398349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2587.726039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.091189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.424956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2588.778069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.102640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.456732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2589.808805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.149310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.481540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2590.865928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.218759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.564749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2591.900560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.267556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2592.646679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.002079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.361566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2593.707227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.043118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.352979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2594.696222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.026548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.367979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2595.688247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.034046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.385530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2596.708292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.031356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.371632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2597.730130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.074231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.419198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2598.744253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.106577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.491641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.873803] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2599.876596] Lustre: Skipped 173 previous similar messages [ 2599.882836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2599.889521] Lustre: Skipped 173 previous similar messages [ 2599.892120] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2599.894812] Lustre: Skipped 173 previous similar messages [ 2599.897356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2599.902468] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2599.905886] Lustre: Skipped 173 previous similar messages [ 2600.250301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.625327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2600.981091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.327634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2601.685571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.034326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.373734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2602.714860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.053775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.400808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2603.780454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.165395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.562718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2604.970242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.352342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2605.747328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.105645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.471503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2606.859829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.213259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.577121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2607.953477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.334756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2608.740703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.132286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.518629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2609.935189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.293450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2610.663349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.024307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.355975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2611.705619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.047922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.401536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2612.759122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.155198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.512856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2613.856641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.206090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.567310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2614.946393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.298362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2615.679911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.089312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.484941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2616.882498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.266618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2617.679645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.096901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.509234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2618.883252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.229747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.588904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2619.926200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.251392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.585750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2620.918887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.229958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.552554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2621.887385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.241286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.589262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2622.942810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.299890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2623.651854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.021232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.402437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2624.757721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.118114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.470213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2625.840327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.168607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.526933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2626.864340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.200268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.551832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2627.874849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.215396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.556141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2628.913977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.297032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2629.682421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.100282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.503827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2630.899814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.325634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2631.723485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.108036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.483757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2632.857386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.250929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2633.680448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.085267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.500898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2634.895667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.355570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2635.767474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.178398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.539587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2636.889571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.255052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2637.634953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.005502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.400591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2638.818852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.290242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2639.728699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.173962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2640.651297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.137050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2641.560794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.027610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.482366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2642.893105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.280365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2643.723606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.131364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.514718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2644.985227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.434036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2645.872664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.298891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2646.750188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.212784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2647.711111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.133461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2648.605549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.067396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.549416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2649.966368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.473459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2650.935606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.349685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2651.810961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.207774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2652.629619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.063129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.475639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2653.895943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.347883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2654.771429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.181650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2655.599124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.034443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.492914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2656.984042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.392348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2657.814648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.197855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.563596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2658.926967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.335227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2659.705634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.074858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.445690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2660.866700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.240081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.598687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2661.974697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.337385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2662.682094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.041458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.393555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2663.768245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.129949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.497173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2664.846195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.199076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.565467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2665.991960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.345949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2666.710053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.085778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.437437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2667.800408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.190535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.572176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2668.949658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.330919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2669.685188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.081331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.467830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2670.872736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.279103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2671.629393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.031507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.409454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2672.744761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.086350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.465559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2673.828757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.205442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.587269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2674.951280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.321150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2675.732608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.129766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.517047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2676.898392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.266730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.629318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2677.965727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.309729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.656769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2678.989204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.341144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2679.680475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.009442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.340548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2680.700589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.061561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.408690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2681.790190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.159643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.502288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2682.858301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.197060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.535460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2683.889647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.218908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.587323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2684.974282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.336870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2685.763518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.179452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.606290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2686.979941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.344760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2687.686777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.067255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.495056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2688.842494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.205832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.548038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2689.903586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.288040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.639813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2690.973320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.284040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.618630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2691.964208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.279410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.601878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2692.964616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.314021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2693.659372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.016098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.365457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2694.703121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.031315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.349376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2695.677686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.020635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.347735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2696.710551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.077412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.404455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2697.730652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.078873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.420345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2698.760092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.090101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.449493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2699.798968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.183661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.557807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2700.913510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.280895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2701.688474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.120593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.525327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2702.874824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.226968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.578409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2703.913416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.269283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.603405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2704.946677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.288129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.609226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2705.948397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.297586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2706.648318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.032356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.394415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2707.737686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.110379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.459908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2708.793398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.158939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.527079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2709.889096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.225259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.529746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2710.837842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.173636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.507110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2711.855782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.246135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2712.677160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.112886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.484667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2713.897757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.269237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2714.625444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.022652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.391097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2715.776965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.167357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.526923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2716.869510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.252306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.583540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2717.915169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.236827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.590469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2718.928866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.242190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.562845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2719.884446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.206609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.523647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2720.861271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.219356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.550613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2721.921692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.328905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2722.701172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.071754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.439136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2723.802746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.148221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.516593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2724.930092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.353615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2725.784487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.225833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2726.612577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.001066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.417480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2727.820661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.203774] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 2728.206483] Lustre: Skipped 341 previous similar messages [ 2728.212657] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2728.217112] Lustre: Skipped 341 previous similar messages [ 2728.219212] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 2728.221400] Lustre: Skipped 341 previous similar messages [ 2728.223198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.227809] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 2728.231146] Lustre: Skipped 341 previous similar messages [ 2728.605060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2728.953046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.296227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2729.658225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.017102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.373285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2730.717442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.072189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.425723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2731.792551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.176238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.583328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2732.967055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.353483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2733.714307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.077054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.448127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2734.798384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.158713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.523322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2735.843703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.169133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.518107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2736.858314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.185208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.509764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2737.840172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.177243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.509664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2738.824273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.161818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.491468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2739.817029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.157988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.486066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2740.821320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.157699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.484865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2741.823670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.160988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.464761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2742.807457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.167128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.499894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2743.824442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.162995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.512453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2744.847170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.186345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.518077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2745.867335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.197306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.520830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2746.850207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.176261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.505466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2747.853925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.182563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.506511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2748.837061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.161531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.484623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2749.823448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.167460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.509180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2750.829437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.156923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.492851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2751.827991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.158495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.477546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2752.794797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.133439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.482698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2753.814550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.151657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.478669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2754.812791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.159815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.526153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2755.861142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.208476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.551714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2756.890278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.213967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.518058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2757.832702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.157809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.506539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2758.838958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.167886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.492802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2759.862060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.216978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.551954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2760.886624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.206408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.521828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2761.830621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.152917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.493100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2762.840069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.225739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.602028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2763.947359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.283370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2764.654455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.008941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.356196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2765.702575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.030945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.362035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2766.693450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.045500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.387781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2767.735063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.089251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.489806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2768.866156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.233066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.617673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2769.979674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.325496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2770.694175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.041758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.421983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2771.763465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.117552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.461787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2772.869481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.239601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.582752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2773.960079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.366839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2774.727902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.124397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.510760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2775.928441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.348191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2776.722207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.126181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.517049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2777.916581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.444876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2778.961038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.397556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2779.854782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.276245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2780.707268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.112640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2781.553541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.005728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2782.531095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.002977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.422361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2783.854451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.255214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2784.645632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.016946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.410937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2785.811815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.192972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2786.606136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.003903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.375519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2787.774910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.172312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2788.595105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.019449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.439814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2789.802839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.189108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.576911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2790.939896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.299847] LustreError: 171450:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2791.304801] LustreError: 171450:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2791.308883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2791.660355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.027530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.354192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2792.703813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.051900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.388488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2793.727557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2794.071613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.502979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2799.887539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.268903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2800.647500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.030284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.429479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2801.868104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.277350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2802.686637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.087744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.490749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2803.976379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.425352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2804.845907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.243888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2805.691722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.091530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.511324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2806.897379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.261073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2807.630449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.001060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.394900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2808.819338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.282341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2809.679850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.060269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.434123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2810.813196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.178932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.546387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2811.905108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.245322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.615733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2812.969860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.322355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2813.675977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.038993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.434573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2814.875708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.297924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2815.692899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.104122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.486414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2816.896341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.348327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2817.777590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.206818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2818.614647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.018604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.405954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2819.816640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.181666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.546728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2820.910308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.275325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2821.642938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.019777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.392664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2822.784552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.184627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.577121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2823.983483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.399216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2824.814170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.197141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.589551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2825.975925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.344909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2826.714826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.096052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.460324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2827.831077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.205731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.577495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2828.967627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.333439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2829.745510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.103042] LustreError: 171450:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 2830.107503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.107587] LustreError: 171450:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 2830.495248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2830.882341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.242738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.586510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2831.931458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.278428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2832.651815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.003487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.347673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2833.691023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.053356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.436572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2834.792888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.161659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.534470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2835.868683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.212828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.545926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2836.919268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.270460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.625513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2837.997043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.361058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2838.709835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.099540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.443773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2839.773240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.108824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.471476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2840.824370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.207635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.607096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2841.956055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.290561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2842.640956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.012301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.350830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2843.695162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.043408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.405082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2844.761936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.131088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.483320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2845.851323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.197164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.555621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2846.902628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.252842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.616488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2847.946939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2848.274931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 2999.213214] LustreError: 173429:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 2999.217460] LustreError: 171450:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 3149.806352] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 3149.811356] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3149.811611] LustreError: Skipped 1 previous similar message [ 3149.816857] Lustre: Skipped 313 previous similar messages [ 3149.819274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3150.816953] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3150.817298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3150.823097] LustreError: Skipped 1 previous similar message [ 3153.560415] Lustre: server umount lustre-OST0000 complete [ 3155.423822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3155.427780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3155.430382] Lustre: Skipped 1 previous similar message [ 3161.058300] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3161.063387] Lustre: Skipped 2 previous similar messages [ 3161.637829] Lustre: server umount lustre-MDT0000 complete [ 3163.648502] LustreError: 171412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866026 with bad export cookie 2524658796462876550 [ 3163.655387] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3163.657555] LustreError: 171412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3163.871349] Lustre: server umount lustre-MDT0001 complete [ 3169.515508] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 3171.067320] Key type lgssc unregistered [ 3171.253833] LNet: 226502:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3171.259708] LNetError: 226502:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3171.269756] LNet: Removed LNI 192.168.206.146@tcp [ 3171.780190] Key type .llcrypt unregistered [ 3171.782167] Key type ._llcrypt unregistered [ 3184.067988] Key type ._llcrypt registered [ 3184.069930] Key type .llcrypt registered [ 3184.130230] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3184.556131] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3184.624306] alg: No test for adler32 (adler32-zlib) [ 3185.562369] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 3185.711181] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 3187.327575] Key type lgssc registered [ 3187.973314] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3191.799588] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3191.816993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3193.059095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3195.127615] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3198.781925] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3198.968990] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3200.683556] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3203.739872] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3205.909668] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3208.757970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3208.892279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3211.199535] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3214.011298] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3218.153257] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 3221.130670] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3221.786679] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3221.792145] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3221.795141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.800071] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3221.800088] LustreError: 227927:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 3223.522264] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3223.523332] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3223.526633] Lustre: Skipped 1 previous similar message [ 3224.551634] Lustre: lustre-OST0000: Not available for connect from 192.168.206.46@tcp (stopping) [ 3224.554585] Lustre: Skipped 1 previous similar message [ 3224.599784] Lustre: server umount lustre-OST0000 complete [ 3241.879109] LustreError: 227927:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 3253.217828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3253.386100] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3255.838812] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3259.141078] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3263.658443] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.665198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.665809] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3263.675756] Lustre: Skipped 1 previous similar message [ 3265.947300] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 6 sec [ 3268.174299] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff908908a30000.ost_server_uuid 50 [ 3268.730212] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff908908a30000.ost_server_uuid in FULL state after 0 sec [ 3274.210397] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3274.210673] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3274.211130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3274.214354] Lustre: Skipped 1 previous similar message [ 3275.919440] Lustre: server umount lustre-OST0000 complete [ 3279.329142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3279.330154] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3279.333791] Lustre: Skipped 1 previous similar message [ 3279.336291] Lustre: Skipped 1 previous similar message [ 3283.352707] Lustre: server umount lustre-MDT0000 complete [ 3284.448148] LustreError: 227908:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3284.453715] LustreError: 227908:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3284.674529] LustreError: 227888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866147 with bad export cookie 529971618069198245 [ 3284.678135] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3284.679236] LustreError: 227888:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3284.796068] Lustre: server umount lustre-MDT0001 complete [ 3288.441435] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 3289.431634] Key type lgssc unregistered [ 3289.560631] LNet: 233113:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3289.563763] LNetError: 233113:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3289.571491] LNet: Removed LNI 192.168.206.146@tcp [ 3289.916139] Key type .llcrypt unregistered [ 3289.917559] Key type ._llcrypt unregistered [ 3302.256073] Key type ._llcrypt registered [ 3302.257496] Key type .llcrypt registered [ 3302.300386] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3302.695208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3302.711616] alg: No test for adler32 (adler32-zlib) [ 3303.593591] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 3303.710656] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 3305.303169] Key type lgssc registered [ 3305.726067] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3309.452554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3309.466716] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3310.667507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3312.285528] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3315.645177] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3315.803499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3317.448360] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3318.595835] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 3321.189629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3321.349356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3322.409424] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 3323.696978] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3327.156344] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3327.229718] Lustre: lustre-OST0001: new disk, initializing [ 3327.231971] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3327.234901] Lustre: 237125:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3327.281518] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3327.614934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3327.618911] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3327.653571] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3329.581181] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3336.723290] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3340.864895] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 3348.450687] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3348.452061] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3348.455071] Lustre: Skipped 2 previous similar messages [ 3348.457565] Lustre: Skipped 1 previous similar message [ 3353.249070] Lustre: server umount lustre-MDT0000 complete [ 3353.571452] LustreError: 236945:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3353.577403] LustreError: 236945:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3354.569412] LustreError: 234794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866216 with bad export cookie 103166356281752812 [ 3354.572773] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3354.573575] LustreError: 234794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3354.693640] Lustre: server umount lustre-MDT0001 complete [ 3365.903578] Lustre: server umount lustre-OST0000 complete [ 3377.595326] Lustre: server umount lustre-OST0001 complete [ 3385.504074] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3389.348657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3389.500802] LustreError: 240227:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3389.522369] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3390.937491] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3393.886655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3395.495875] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3396.645912] Lustre: 241368:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3399.084311] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3399.137444] LustreError: 241722:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3399.209293] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3399.211945] Lustre: Skipped 1 previous similar message [ 3401.196893] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3404.145191] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3404.223492] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 3406.215017] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3409.433586] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3410.812991] Lustre: 243234:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3415.523140] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3415.525273] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3415.528059] Lustre: Skipped 4 previous similar messages [ 3415.530651] Lustre: Skipped 4 previous similar messages [ 3420.642724] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3420.645607] Lustre: Skipped 1 previous similar message [ 3421.405468] Lustre: server umount mylustre-MDT0000 complete [ 3422.821429] LustreError: 240970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866285 with bad export cookie 103166356281755577 [ 3422.826085] LustreError: 240970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3422.826792] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3422.950942] Lustre: server umount mylustre-MDT0001 complete [ 3434.013283] Lustre: server umount mylustre-OST0000 complete [ 3446.012558] Lustre: server umount mylustre-OST0001 complete [ 3454.377409] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3458.529683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3458.747206] LustreError: 245540:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3458.756065] LustreError: 245540:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3458.790870] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3458.795268] Lustre: Skipped 1 previous similar message [ 3460.679280] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3464.161510] LustreError: 245541:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3464.419349] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3466.338204] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3467.713535] Lustre: 246683:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3470.659262] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3473.196367] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3473.892994] LustreError: 247484:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3477.030475] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3479.658415] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3483.630129] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 3483.631763] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 3488.832350] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3490.327214] Lustre: 248554:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3493.859953] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3493.862236] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 3493.863971] Lustre: Skipped 3 previous similar messages [ 3493.868076] Lustre: Skipped 4 previous similar messages [ 3498.922344] Lustre: server umount tfs-MDT0000 complete [ 3498.979683] LustreError: 245540:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3498.986507] LustreError: 245540:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3500.415385] LustreError: 245515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866362 with bad export cookie 103166356281757243 [ 3500.419249] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3500.421275] LustreError: 245515:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3500.576937] Lustre: server umount tfs-MDT0001 complete [ 3512.330267] Lustre: server umount tfs-OST0000 complete [ 3524.079723] Lustre: server umount tfs-OST0001 complete [ 3533.741597] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3538.939657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3539.196280] LustreError: 250801:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3539.245121] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3539.247800] Lustre: Skipped 3 previous similar messages [ 3541.395167] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3545.379129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3547.303266] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3548.576376] Lustre: 251943:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 3551.406590] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3554.028905] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3557.568823] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3560.157283] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3564.008845] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 3564.008850] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 3569.159676] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 3570.499529] Lustre: 253812:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 3573.214980] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 07:20:35 (1788866435) [ 3586.528362] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3586.532943] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3586.538333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3591.593259] Lustre: server umount lustre-MDT0000 complete [ 3593.152276] LustreError: 250776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866455 with bad export cookie 103166356281758909 [ 3593.156257] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3593.159400] LustreError: 250776:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3597.795449] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3597.809599] Lustre: Skipped 3 previous similar messages [ 3597.816031] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 3597.825264] Lustre: Skipped 5 previous similar messages [ 3599.397724] Lustre: server umount lustre-MDT0001 complete [ 3607.311047] Lustre: server umount lustre-OST0000 complete [ 3623.391109] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3623.469060] Lustre: server umount lustre-OST0001 complete [ 3626.166388] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 3630.058653] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3646.154908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3646.272745] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3646.290151] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3646.345938] Lustre: lustre-MDT0000: new disk, initializing [ 3646.392738] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 3646.396760] Lustre: Skipped 3 previous similar messages [ 3646.405958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3648.538799] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3654.753993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3654.852305] Lustre: 256988:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3654.894168] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3654.896705] Lustre: Skipped 1 previous similar message [ 3654.945687] Lustre: lustre-MDT0001: new disk, initializing [ 3655.041779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3655.047606] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3657.139960] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3662.486703] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3662.626753] Lustre: lustre-OST0000: new disk, initializing [ 3662.629177] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3662.634261] Lustre: 258571:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3663.819354] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3663.824058] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3663.873893] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3665.305738] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3670.806620] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3670.886633] Lustre: lustre-OST0001: new disk, initializing [ 3670.890075] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3670.894441] Lustre: 259440:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3672.011110] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3672.016656] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3672.067330] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3673.358822] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3678.839546] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 07:22:21 (1788866541) [ 3684.091528] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 07:22:26 (1788866546) [ 3697.636300] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3697.638219] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3697.641881] Lustre: Skipped 2 previous similar messages [ 3697.644663] Lustre: Skipped 2 previous similar messages [ 3701.963581] Lustre: server umount lustre-OST0000 complete [ 3702.758346] LustreError: 258554:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3702.769361] LustreError: 258554:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 11 previous similar messages [ 3706.336883] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3709.808694] Lustre: server umount lustre-MDT0000 complete [ 3711.374634] LustreError: 256980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866573 with bad export cookie 103166356281760463 [ 3711.376943] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3711.383034] LustreError: 256980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3717.583715] Lustre: server umount lustre-MDT0001 complete [ 3721.984766] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 3793.441094] Lustre: server umount lustre-OST0001 complete [ 3795.954476] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 3802.819369] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3817.792020] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3817.910231] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3817.924738] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3817.968896] Lustre: lustre-MDT0000: new disk, initializing [ 3818.012205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3818.019080] Lustre: Skipped 3 previous similar messages [ 3818.027439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3819.948783] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3825.089655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3825.153884] Lustre: 264261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3825.291072] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3827.168333] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3832.035750] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3834.779219] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3838.159342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3838.288738] Lustre: lustre-OST0000: new disk, initializing [ 3838.291305] Lustre: Skipped 1 previous similar message [ 3838.295276] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3838.298768] Lustre: Skipped 2 previous similar messages [ 3838.301373] Lustre: 266231:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3839.882114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3839.886813] Lustre: Skipped 1 previous similar message [ 3839.890546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3839.942359] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3841.019946] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3845.455265] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3850.210936] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3850.213354] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3850.218153] LustreError: Skipped 1 previous similar message [ 3850.218801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3850.222507] Lustre: Skipped 5 previous similar messages [ 3850.230493] Lustre: Skipped 8 previous similar messages [ 3854.506294] Lustre: server umount lustre-OST0000 complete [ 3856.353057] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3862.239364] Lustre: server umount lustre-MDT0000 complete [ 3863.762306] LustreError: 265056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866726 with bad export cookie 103166356281761933 [ 3863.765648] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3863.766594] LustreError: 265056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3863.900572] Lustre: server umount lustre-MDT0001 complete [ 3868.376898] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 3869.587491] Key type lgssc unregistered [ 3869.729563] LNet: 267985:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3869.733353] LNetError: 267985:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3869.744866] LNet: Removed LNI 192.168.206.146@tcp [ 3870.109131] Key type .llcrypt unregistered [ 3870.110602] Key type ._llcrypt unregistered [ 3881.322037] Key type ._llcrypt registered [ 3881.323751] Key type .llcrypt registered [ 3881.380566] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3881.855258] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3881.868560] alg: No test for adler32 (adler32-zlib) [ 3882.759418] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 3882.867317] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 3884.463183] Key type lgssc registered [ 3884.955488] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3888.255488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3888.265136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3889.447782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3891.196228] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3894.592492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3894.771642] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3896.437593] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3899.369645] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3901.771714] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3904.768592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3904.948538] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3907.794145] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 3911.592583] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3912.156814] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 3914.085077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3914.087267] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3919.220309] Lustre: server umount lustre-OST0000 complete [ 3924.448921] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3924.450015] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3924.458283] Lustre: Skipped 1 previous similar message [ 3924.461707] Lustre: Skipped 1 previous similar message [ 3927.271981] Lustre: server umount lustre-MDT0000 complete [ 3928.855035] LustreError: 269399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866791 with bad export cookie 1219481500661706758 [ 3928.857435] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3928.863804] LustreError: 269399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3929.009351] Lustre: server umount lustre-MDT0001 complete [ 3933.643857] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 3934.970922] Key type lgssc unregistered [ 3935.147417] LNet: 272999:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3935.151559] LNetError: 272999:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3935.159782] LNet: Removed LNI 192.168.206.146@tcp [ 3935.564669] Key type .llcrypt unregistered [ 3935.566817] Key type ._llcrypt unregistered [ 3943.648947] Key type ._llcrypt registered [ 3943.650464] Key type .llcrypt registered [ 3943.701377] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 3944.144414] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3944.161961] alg: No test for adler32 (adler32-zlib) [ 3945.040335] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 3945.151055] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 3946.751293] Key type lgssc registered [ 3947.185306] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3950.629795] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3950.639076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3966.047359] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3971.167527] LustreError: 274166:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.146@tcp: failed processing log, type 4: rc = -110 [ 3997.793673] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4000.102680] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4003.148057] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4004.036326] Lustre: server umount lustre-OST0000 complete [ 4010.422844] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 4011.544208] Key type lgssc unregistered [ 4011.681422] LNet: 275793:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4011.684585] LNetError: 275793:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4011.692429] LNet: Removed LNI 192.168.206.146@tcp [ 4012.030243] Key type .llcrypt unregistered [ 4012.031810] Key type ._llcrypt unregistered [ 4028.350956] Key type ._llcrypt registered [ 4028.353234] Key type .llcrypt registered [ 4028.404992] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4028.760585] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4028.768498] alg: No test for adler32 (adler32-zlib) [ 4029.643492] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 4029.754400] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 4031.351268] Key type lgssc registered [ 4031.803793] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4037.304280] loop0: detected capacity change from 0 to 204800000 [ 4037.427109] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4037.433488] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4037.587572] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.041694] loop0: detected capacity change from 0 to 204800000 [ 4038.165027] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.170536] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.332739] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.756835] loop0: detected capacity change from 0 to 204800000 [ 4038.837405] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.845857] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4038.897746] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4039.347378] loop0: detected capacity change from 0 to 204800000 [ 4039.410893] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 4039.878795] loop0: detected capacity change from 0 to 204800000 [ 4039.891584] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4041.531476] loop0: detected capacity change from 0 to 204800000 [ 4041.542054] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4043.298818] loop0: detected capacity change from 0 to 204800000 [ 4043.310200] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4045.014306] loop0: detected capacity change from 0 to 204800000 [ 4045.023439] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 4046.564712] loop0: detected capacity change from 0 to 204800000 [ 4046.595932] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4046.608189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4046.646087] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4046.650977] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 4046.656526] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4046.661606] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 4046.665921] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4046.670572] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4046.676387] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 4046.681839] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 4046.686836] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 4046.692669] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 4046.697451] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 4046.702718] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 4046.707709] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 4046.713081] Lustre: 279176:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 4049.560095] Lustre: server umount MGS complete [ 4049.911736] loop0: detected capacity change from 0 to 204800000 [ 4049.942194] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4050.125595] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4050.144689] Lustre: lustre-MDT0000: Aborting client recovery [ 4050.512409] loop1: detected capacity change from 0 to 204800000 [ 4050.548394] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4050.615598] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4050.621757] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 4050.627336] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 4050.632313] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4050.636790] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4050.642305] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 4050.647445] Lustre: 279607:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4050.656122] Lustre: Found index 1 for lustre-MDT0001, updating log [ 4050.732513] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4050.763115] Lustre: lustre-MDT0001: Aborting client recovery [ 4051.147174] loop2: detected capacity change from 0 to 204800000 [ 4051.210238] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4051.238381] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4051.243847] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 4051.249106] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 4051.254181] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4051.259374] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 4051.264745] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4051.269791] Lustre: 279687:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4051.322330] Lustre: Found index 0 for lustre-OST0000, updating log [ 4051.336034] Lustre: 279706:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4051.367564] Lustre: lustre-OST0000: Aborting recovery [ 4051.769352] loop3: detected capacity change from 0 to 204800000 [ 4051.805086] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4051.837660] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 4051.843447] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 4051.848682] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 4051.856801] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 4051.868456] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 4051.875319] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 4051.884573] Lustre: 279771:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 4051.916542] Lustre: 279776:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4051.949671] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4051.952808] Lustre: Skipped 1 previous similar message [ 4051.961982] Lustre: lustre-OST0001: Aborting recovery [ 4057.064299] LustreError: 279551:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -108 [ 4060.129087] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4060.137343] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 4060.139719] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 4060.148057] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 4061.154576] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 4061.154732] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 4063.192444] LustreError: 279517:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 341us: rc = -2 [ 4076.512035] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4076.512180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4076.512404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4076.516888] Lustre: Skipped 1 previous similar message [ 4080.991591] Lustre: server umount lustre-MDT0000 complete [ 4081.320260] LustreError: 279501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788866943 with bad export cookie 8803266429014612946 [ 4081.321313] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4081.327588] LustreError: 279501:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4081.631516] LustreError: 279516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4081.631593] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4081.631830] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4081.631835] Lustre: Skipped 3 previous similar messages [ 4081.638624] LustreError: 279516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 4081.644091] Lustre: Skipped 3 previous similar messages [ 4086.751780] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 4086.751782] LustreError: 279515:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4086.760313] Lustre: Skipped 2 previous similar messages [ 4087.540568] Lustre: server umount lustre-MDT0001 complete [ 4102.111184] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4102.152104] Lustre: server umount lustre-OST0000 complete [ 4108.693185] Lustre: server umount lustre-OST0001 complete [ 4111.817934] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 07:29:34 (1788866974) [ 4120.703500] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4151.330819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4151.429326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4151.443452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4151.483723] Lustre: lustre-MDT0000: new disk, initializing [ 4151.513406] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4151.522671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4153.122976] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4156.541110] Lustre: server umount lustre-MDT0000 complete [ 4163.123139] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 07:30:25 (1788867025) [ 4174.206345] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 4178.325318] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4193.096458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4193.197825] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4193.216327] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4193.218535] Lustre: Skipped 1 previous similar message [ 4193.255694] Lustre: lustre-MDT0000: new disk, initializing [ 4193.290573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4193.302694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4194.957414] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4199.783197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4199.834161] Lustre: 286870:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4199.861906] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4199.864266] Lustre: Skipped 1 previous similar message [ 4199.901763] Lustre: lustre-MDT0001: new disk, initializing [ 4199.963326] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4199.970563] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4201.577477] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4205.504880] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4207.551346] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4210.237980] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4210.357773] Lustre: lustre-OST0000: new disk, initializing [ 4210.359807] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4210.363516] Lustre: 288836:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4210.401979] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4210.405201] Lustre: Skipped 1 previous similar message [ 4211.718587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4211.722728] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4211.764250] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4212.835331] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4217.351669] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4220.080215] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4232.350991] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4238.715024] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4283.361697] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4283.363540] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4283.372181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4283.374422] Lustre: Skipped 1 previous similar message [ 4288.482163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4288.484486] Lustre: Skipped 1 previous similar message [ 4296.159144] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4296.224702] Lustre: server umount lustre-OST0000 complete [ 4298.720757] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4298.721420] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4298.724250] Lustre: Skipped 2 previous similar messages [ 4298.727858] Lustre: Skipped 3 previous similar messages [ 4303.651312] Lustre: server umount lustre-MDT0000 complete [ 4303.840121] LustreError: 286883:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4303.853644] LustreError: 286883:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4305.246711] LustreError: 286864:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867167 with bad export cookie 8803266429014615900 [ 4305.250684] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4305.251488] LustreError: 286864:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4305.404718] Lustre: server umount lustre-MDT0001 complete [ 4308.806812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4310.473637] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4312.296522] Lustre: server umount MGS complete [ 4316.712734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4316.950888] LustreError: 292990:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4316.972375] LustreError: 293003:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4317.009636] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4318.651765] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4322.006689] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4323.925685] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4327.104962] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4329.550746] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4332.232970] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4334.807139] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4337.796898] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4362.212257] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4362.212755] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4362.213745] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4362.218392] Lustre: Skipped 1 previous similar message [ 4362.226910] LustreError: Skipped 1 previous similar message [ 4366.750240] Lustre: server umount lustre-OST0000 complete [ 4368.352188] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4372.449931] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4372.455054] Lustre: Skipped 1 previous similar message [ 4374.365735] Lustre: server umount lustre-MDT0000 complete [ 4375.851442] LustreError: 293725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867238 with bad export cookie 8803266429014617741 [ 4375.853332] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4375.858694] LustreError: 293725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4376.021526] Lustre: server umount lustre-MDT0001 complete [ 4380.466185] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 4381.607353] Key type lgssc unregistered [ 4381.755171] LNet: 296930:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4381.758705] LNetError: 296930:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4381.766428] LNet: Removed LNI 192.168.206.146@tcp [ 4382.144304] Key type .llcrypt unregistered [ 4382.145723] Key type ._llcrypt unregistered [ 4397.171560] Key type ._llcrypt registered [ 4397.173162] Key type .llcrypt registered [ 4397.212598] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 4404.102427] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4404.591025] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4404.601270] alg: No test for adler32 (adler32-zlib) [ 4405.477713] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 4405.588422] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 4407.183159] Key type lgssc registered [ 4407.658492] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4422.509152] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4422.518749] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4423.630310] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4423.650159] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4423.690681] Lustre: lustre-MDT0000: new disk, initializing [ 4423.722098] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4423.730040] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4425.347782] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4429.903423] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4429.954670] Lustre: 300856:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4429.986727] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4429.989647] Lustre: Skipped 1 previous similar message [ 4430.029570] Lustre: lustre-MDT0001: new disk, initializing [ 4430.071657] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4430.087111] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4430.090920] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4431.650253] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4435.889911] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4438.038676] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4440.844180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4440.958091] Lustre: lustre-OST0000: new disk, initializing [ 4440.960713] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4440.964777] Lustre: 302824:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4440.999225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4441.163466] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4441.168534] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4441.209984] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4443.293869] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4447.641360] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4452.354325] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 4463.267665] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 4468.072489] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 4473.625784] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 4518.369313] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4518.370580] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4518.374934] LustreError: Skipped 1 previous similar message [ 4518.379808] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4523.490581] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4523.494073] Lustre: Skipped 2 previous similar messages [ 4523.550379] Lustre: server umount lustre-OST0000 complete [ 4527.584711] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4527.588261] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4527.593326] Lustre: Skipped 1 previous similar message [ 4527.596027] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4528.610241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4531.488674] Lustre: server umount lustre-MDT0000 complete [ 4533.108339] LustreError: 300849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867395 with bad export cookie 18323753044193308525 [ 4533.112620] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4533.113688] LustreError: 300849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4533.261696] Lustre: server umount lustre-MDT0001 complete [ 4536.831433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4538.556199] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4540.479753] Lustre: server umount MGS complete [ 4544.917546] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4545.131342] LustreError: 306891:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 4545.165963] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4546.769920] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4550.069938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4550.226338] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4551.987678] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4555.066246] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4557.115906] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4559.741076] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4559.895747] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4562.080427] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4564.708192] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4596.705704] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4596.707521] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4596.708908] LustreError: Skipped 1 previous similar message [ 4596.715232] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4596.718566] Lustre: Skipped 3 previous similar messages [ 4600.724539] Lustre: server umount lustre-OST0000 complete [ 4606.432112] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4606.435203] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4606.438994] Lustre: Skipped 1 previous similar message [ 4606.441058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4608.307094] Lustre: server umount lustre-MDT0000 complete [ 4609.757187] LustreError: 306858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867472 with bad export cookie 18323753044193310366 [ 4609.760865] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4609.762297] LustreError: 306858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4609.887569] Lustre: server umount lustre-MDT0001 complete [ 4614.238985] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 4615.499333] Key type lgssc unregistered [ 4615.649522] LNet: 310844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4615.653395] LNetError: 310844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4615.664408] LNet: Removed LNI 192.168.206.146@tcp [ 4616.037184] Key type .llcrypt unregistered [ 4616.038573] Key type ._llcrypt unregistered [ 4623.863602] Key type ._llcrypt registered [ 4623.865015] Key type .llcrypt registered [ 4623.906676] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 4630.282322] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4630.736868] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4630.742705] alg: No test for adler32 (adler32-zlib) [ 4631.603840] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 4631.699476] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 4633.287202] Key type lgssc registered [ 4633.703896] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4643.037170] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 4648.616598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4648.623782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4649.713388] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4649.726187] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4649.767350] Lustre: lustre-MDT0000: new disk, initializing [ 4649.811640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4649.819693] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4651.346381] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4655.990217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4656.110792] Lustre: lustre-OST0000: new disk, initializing [ 4656.114637] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4656.116902] Lustre: Skipped 1 previous similar message [ 4656.119346] Lustre: 314874:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4656.148237] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4658.188257] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4658.192077] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4658.205218] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4658.391565] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 4897.798006] hrtimer: interrupt took 1401563 ns [ 5031.905099] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5031.913573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5037.023957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5037.717798] Lustre: server umount lustre-MDT0000 complete [ 5040.441985] LustreError: 315869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867902 with bad export cookie 2633992496116466384 [ 5040.442569] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5040.454391] LustreError: 315869:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5040.599084] Lustre: server umount lustre-OST0000 complete [ 5056.174501] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 5061.857715] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5080.002253] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5080.150470] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5080.172199] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5080.221434] Lustre: lustre-MDT0000: new disk, initializing [ 5080.273412] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5080.283818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5082.733308] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5088.358292] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5088.419615] Lustre: 319135:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5088.462756] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5088.465380] Lustre: Skipped 1 previous similar message [ 5088.502866] Lustre: lustre-MDT0001: new disk, initializing [ 5088.548548] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5088.569532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5088.574096] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5090.688935] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5095.884914] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5098.601645] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5102.011038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5102.182369] Lustre: lustre-OST0000: new disk, initializing [ 5102.185412] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5102.188915] Lustre: 321102:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5102.246389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5105.525986] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5109.271230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5109.282056] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5109.342971] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5110.900302] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5113.301852] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5113.414633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5115.948377] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5116.071635] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5119.463709] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5119.465293] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5119.471244] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5119.477051] LustreError: Skipped 1 previous similar message [ 5119.480227] Lustre: Skipped 1 previous similar message [ 5123.380431] Lustre: server umount lustre-OST0000 complete [ 5129.697312] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5129.700153] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5129.700965] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5129.703664] Lustre: Skipped 2 previous similar messages [ 5131.397858] Lustre: server umount lustre-MDT0000 complete [ 5133.243772] LustreError: 319128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788867995 with bad export cookie 2633992496119407161 [ 5133.246554] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5133.251787] LustreError: 319128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5133.425476] Lustre: server umount lustre-MDT0001 complete [ 5135.043466] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 5135.868569] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 07:46:38 (1788867998) [ 5139.112955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5139.340442] LustreError: 323410:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5139.348896] LustreError: 323410:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5139.371771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5141.002532] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5144.534199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5144.544141] LustreError: 323411:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5146.335511] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5149.480540] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5151.842543] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5154.909173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5157.708674] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5161.004613] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5164.140533] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5164.237501] Lustre: lustre-OST0001: new disk, initializing [ 5164.240434] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5164.244782] Lustre: 326241:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5164.289858] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5164.294642] Lustre: Skipped 2 previous similar messages [ 5164.485522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5164.493130] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5164.516463] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5165.546722] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5165.551956] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5166.562090] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5166.569162] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 5166.885136] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5168.611541] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5168.616610] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 5171.419203] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5171.519973] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5172.705747] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 5172.709586] LustreError: 326277:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 5173.860696] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff908918cc0000.ost_server_uuid 50 [ 5174.568447] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff908918cc0000.ost_server_uuid in FULL state after 0 sec [ 5177.133441] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff908918cc0000.ost_server_uuid 50 [ 5177.761218] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff908918cc0000.ost_server_uuid in FULL state after 0 sec [ 5178.925258] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5190.119777] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5190.124057] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5190.127574] Lustre: Skipped 1 previous similar message [ 5190.133305] Lustre: Skipped 2 previous similar messages [ 5196.189116] Lustre: server umount lustre-OST0001 complete [ 5200.356600] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5200.357797] LustreError: 325231:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5200.363775] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5200.364246] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5200.364257] Lustre: Skipped 4 previous similar messages [ 5200.374099] LustreError: 325231:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5204.524209] Lustre: server umount lustre-OST0000 complete [ 5210.593321] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5210.599727] Lustre: Skipped 2 previous similar messages [ 5212.573431] Lustre: server umount lustre-MDT0000 complete [ 5214.150328] LustreError: 323391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868076 with bad export cookie 2633992496119408043 [ 5214.153385] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5214.155265] LustreError: 323391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5214.312280] Lustre: server umount lustre-MDT0001 complete [ 5218.595244] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 5219.799333] Key type lgssc unregistered [ 5219.954614] LNet: 328921:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5219.958244] LNetError: 328921:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5219.967588] LNet: Removed LNI 192.168.206.146@tcp [ 5220.379310] Key type .llcrypt unregistered [ 5220.380356] Key type ._llcrypt unregistered [ 5231.674803] Key type ._llcrypt registered [ 5231.676532] Key type .llcrypt registered [ 5231.721709] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5232.145725] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5232.203629] alg: No test for adler32 (adler32-zlib) [ 5233.095079] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 5233.207507] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 5234.807317] Key type lgssc registered [ 5235.207431] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5238.293575] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5238.303691] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5239.495140] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5241.081671] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5244.116510] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5247.162264] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5247.338340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5249.058855] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5253.107127] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5253.274946] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5255.904000] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5257.380860] LustreError: 331993:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5257.390038] LustreError: 331993:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5258.953459] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5261.884568] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5262.019043] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5264.112159] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5264.116196] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 5264.818882] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5268.269536] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5271.401477] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 5271.527895] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5356.517112] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5356.517944] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5356.525807] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5356.532614] Lustre: Skipped 1 previous similar message [ 5361.635502] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 5361.640669] Lustre: Skipped 1 previous similar message [ 5361.800910] Lustre: server umount lustre-OST0001 complete [ 5365.219539] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5365.223760] LustreError: Skipped 1 previous similar message [ 5365.225824] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5365.230747] Lustre: Skipped 1 previous similar message [ 5365.234703] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5366.752950] LustreError: 331994:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5366.756339] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5366.764471] LustreError: 331994:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5366.774298] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5370.090498] Lustre: server umount lustre-OST0000 complete [ 5371.873478] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5371.874327] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5371.881550] Lustre: Skipped 1 previous similar message [ 5371.892157] Lustre: Skipped 2 previous similar messages [ 5376.992498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5376.996375] Lustre: Skipped 1 previous similar message [ 5386.207148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5386.288734] Lustre: server umount lustre-MDT0000 complete [ 5387.232072] LustreError: 330315:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5387.243597] LustreError: 330315:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5387.879207] LustreError: 334001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868250 with bad export cookie 2391083719226385984 [ 5387.880766] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5387.885710] LustreError: 334001:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5388.026147] Lustre: server umount lustre-MDT0001 complete [ 5392.416404] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 5393.608221] Key type lgssc unregistered [ 5393.749514] LNet: 335254:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5393.753543] LNetError: 335254:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5393.762945] LNet: Removed LNI 192.168.206.146@tcp [ 5394.144623] Key type .llcrypt unregistered [ 5394.146127] Key type ._llcrypt unregistered [ 5415.305683] Key type ._llcrypt registered [ 5415.306638] Key type .llcrypt registered [ 5415.354376] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5415.670624] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5415.679512] alg: No test for adler32 (adler32-zlib) [ 5416.568651] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 5416.679595] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 5418.287179] Key type lgssc registered [ 5418.795516] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5423.096326] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5423.106383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5424.341977] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5426.252376] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5430.572996] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5430.782198] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5432.961529] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5434.403608] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5437.350878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5437.529578] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5440.165927] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5440.615653] LustreError: 339001:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5440.627390] LustreError: 339001:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5444.544654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5444.694883] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5447.578267] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5448.823958] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 5448.824711] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5453.929408] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5458.017617] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5464.546759] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5464.549783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5464.554903] Lustre: Skipped 3 previous similar messages [ 5464.558364] Lustre: Skipped 2 previous similar messages [ 5468.670177] Lustre: server umount lustre-MDT0000 complete [ 5469.667668] LustreError: 337516:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5469.676197] LustreError: 337516:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 5470.329240] LustreError: 338653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868332 with bad export cookie 15076007667864452834 [ 5470.332175] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5470.335941] LustreError: 338653:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5470.488339] Lustre: server umount lustre-MDT0001 complete [ 5481.996781] Lustre: server umount lustre-OST0000 complete [ 5492.809291] Lustre: server umount lustre-OST0001 complete [ 5500.986167] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5505.849025] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5506.052224] LustreError: 342773:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5506.087357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5507.793754] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5511.137024] LustreError: 342774:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5511.357921] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5513.118662] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5514.401676] Lustre: 343913:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5517.195696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5517.372029] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5517.375221] Lustre: Skipped 1 previous similar message [ 5519.888807] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5521.449481] LustreError: 344269:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5523.766065] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5526.714726] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5531.115542] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5531.117501] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 5535.915352] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5537.420488] Lustre: 345783:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5541.344767] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5541.351364] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5541.352784] Lustre: Skipped 2 previous similar messages [ 5541.359967] Lustre: Skipped 4 previous similar messages [ 5545.136898] Lustre: server umount lustre-MDT0000 complete [ 5546.464533] LustreError: 342770:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5546.471891] LustreError: 342770:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 5546.778957] LustreError: 342755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868409 with bad export cookie 15076007667864454521 [ 5546.785399] LustreError: 342755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5546.786941] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5546.932623] Lustre: server umount lustre-MDT0001 complete [ 5558.802148] Lustre: server umount lustre-OST0000 complete [ 5571.114119] Lustre: server umount lustre-OST0001 complete [ 5572.716258] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 5573.413407] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 5574.137155] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 07:53:56 (1788868436) [ 5583.992528] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5588.276805] loop0: detected capacity change from 0 to 1099511627776 [ 5588.495237] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 5588.499322] *** VALIDATE xfs *** [ 5588.505095] XFS (loop0): Mounting V5 Filesystem [ 5588.519327] XFS (loop0): Ending clean mount [ 5589.112482] loop1: detected capacity change from 0 to 18691697672192 [ 5622.761712] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 5627.509062] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5644.764712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5644.864929] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5644.878063] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5644.917115] Lustre: lustre-MDT0000: new disk, initializing [ 5644.950917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5644.953930] Lustre: Skipped 1 previous similar message [ 5644.963333] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5646.845322] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5652.459891] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5652.524170] Lustre: 350899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5652.556610] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5652.560582] Lustre: Skipped 1 previous similar message [ 5652.603648] Lustre: lustre-MDT0001: new disk, initializing [ 5652.695300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5652.700861] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5654.579280] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5659.238473] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5661.645497] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5664.627147] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5664.774381] Lustre: lustre-OST0000: new disk, initializing [ 5664.777552] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5664.781973] Lustre: 352864:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5665.927408] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5665.934829] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5665.984306] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5667.429991] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5671.742674] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5673.652223] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 5673.732445] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 5675.586218] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 5675.677592] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 5676.513409] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5676.513503] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5676.515482] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5676.519373] Lustre: Skipped 2 previous similar messages [ 5676.528140] LustreError: Skipped 1 previous similar message [ 5681.633291] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5681.636580] Lustre: Skipped 2 previous similar messages [ 5682.599146] Lustre: server umount lustre-OST0000 complete [ 5686.753443] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5686.756291] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5686.767365] Lustre: Skipped 1 previous similar message [ 5690.468352] Lustre: server umount lustre-MDT0000 complete [ 5691.872421] LustreError: 350909:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5691.878452] LustreError: 350909:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 5692.146214] LustreError: 352863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868554 with bad export cookie 15076007667864456054 [ 5692.148773] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5692.151888] LustreError: 352863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5692.288784] Lustre: server umount lustre-MDT0001 complete [ 5696.008635] XFS (loop0): Unmounting Filesystem [ 5698.178593] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 07:56:00 (1788868560) [ 5701.020947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5701.240582] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5701.242976] Lustre: Skipped 2 previous similar messages [ 5702.751688] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5705.868024] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5707.652286] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5710.845758] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5713.269794] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5716.523175] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5719.322261] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5722.458130] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5736.418517] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5736.420064] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5736.420438] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5736.426234] Lustre: Skipped 1 previous similar message [ 5736.428211] Lustre: Skipped 1 previous similar message [ 5737.050418] Lustre: server umount lustre-OST0000 complete [ 5740.510860] Lustre: server umount lustre-MDT0000 complete [ 5741.537438] LustreError: 355419:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5741.545129] LustreError: 355419:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 5742.122717] LustreError: 355401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788868604 with bad export cookie 15076007667864456936 [ 5742.127148] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5742.128637] LustreError: 355401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5742.301496] Lustre: server umount lustre-MDT0001 complete [ 5746.456882] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 5747.675910] Key type lgssc unregistered [ 5747.844510] LNet: 359064:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5747.849614] LNetError: 359064:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5747.860470] LNet: Removed LNI 192.168.206.146@tcp [ 5748.262132] Key type .llcrypt unregistered [ 5748.263862] Key type ._llcrypt unregistered [ 5760.088978] Key type ._llcrypt registered [ 5760.090672] Key type .llcrypt registered [ 5760.134608] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 5760.521199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5760.565325] alg: No test for adler32 (adler32-zlib) [ 5761.493792] Lustre: Lustre: Build Version: 2.17.58_39_g331ba7d [ 5761.634291] LNet: Added LNI 192.168.206.146@tcp [8/256/0/180] [ 5763.231780] Key type lgssc registered [ 5763.782342] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5767.150774] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5767.160219] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5768.346489] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5769.983231] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5773.690339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5773.929424] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5775.878787] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5779.165412] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5781.436067] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5784.257388] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5784.407804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5786.681401] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5789.629468] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5804.003307] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5804.003496] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 5804.010922] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5804.016248] Lustre: Skipped 1 previous similar message [ 5806.567173] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.46@tcp (stopping) [ 5809.050523] Lustre: server umount lustre-MDT0001 complete [ 5809.122244] LustreError: 360598:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5809.128342] LustreError: 360598:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 5811.675360] LustreError: 360597:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5813.055585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5813.113479] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 5813.116389] Lustre: Found index 1 for lustre-MDT0001, updating log [ 5813.133788] Lustre: 360589:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5813.265393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5815.168917] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 5818.342832] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 5822.953026] LustreError: 364408:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 5822.957691] LustreError: 364408:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 5822.963271] LustreError: 360594:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 5822.988389] LustreError: 364422:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.146@tcp: cfg command failed: rc = -17 [ 5822.992284] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 5822.992284] [ 6118.886407] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.46@tcp (stopping) [ 6120.419125] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6120.419902] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6120.424153] Lustre: Skipped 1 previous similar message [ 6123.996782] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.46@tcp (stopping) [ 6124.004461] Lustre: Skipped 2 previous similar messages [ 6124.843297] Lustre: server umount lustre-MDT0001 complete [ 6125.537923] LustreError: 360602:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6125.548351] LustreError: 360602:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6129.122735] LustreError: 360598:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6129.470051] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6129.554352] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6129.558243] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6129.577130] Lustre: 360589:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6129.730041] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6131.462663] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 6134.761382] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6139.373854] LustreError: 376215:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6139.378633] LustreError: 376215:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6139.382428] LustreError: 360594:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6139.384557] LustreError: 360594:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6139.422258] LustreError: 376218:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 6139.428768] LustreError: 376218:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 6139.431709] LustreError: 376218:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.146@tcp: cfg command failed: rc = -17 [ 6139.435654] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.146@tcp [ 6139.435654] [ 6436.328095] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.46@tcp (stopping) [ 6436.833697] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 6436.833794] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6436.849479] Lustre: Skipped 2 previous similar messages [ 6440.105700] Lustre: server umount lustre-MDT0001 complete [ 6441.439342] LustreError: 382949:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6444.474399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6444.545540] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 6444.549483] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6444.578179] Lustre: 360592:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6444.773137] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 6446.636642] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 6450.150254] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 6452.709466] LustreError: 387936:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 6452.712755] LustreError: 387936:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 6452.716437] LustreError: 360594:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 6452.718982] LustreError: 360594:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 6452.738866] LustreError: 387939:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 6452.745701] LustreError: 387939:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.146@tcp: cfg command failed: rc = -114 [ 6452.750346] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 6452.750346] [ 6762.464879] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6762.471228] Lustre: Skipped 1 previous similar message [ 6762.471440] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6762.474163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6762.479713] Lustre: Skipped 3 previous similar messages [ 6767.823443] Lustre: server umount lustre-MDT0000 complete [ 6769.664325] LustreError: 375032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788869632 with bad export cookie 1959496425448178634 [ 6769.672498] LustreError: 375032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6769.672535] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6772.704051] LustreError: 363092:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6772.712477] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6772.718107] LustreError: 363092:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 6772.722784] Lustre: Skipped 1 previous similar message [ 6775.881449] Lustre: server umount lustre-MDT0001 complete [ 6777.752890] Lustre: server umount lustre-OST0000 complete [ 6782.573523] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 6787.036549] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 6801.636927] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 08:14:24 (1788869664) [ 6804.514705] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6804.618925] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6804.635814] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6804.681355] Lustre: lustre-MDT0000: new disk, initializing [ 6804.716473] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6804.727461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6806.409261] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 6811.684765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6811.742375] Lustre: 401503:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6811.775985] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6811.779414] Lustre: Skipped 1 previous similar message [ 6811.829923] Lustre: lustre-MDT0001: new disk, initializing [ 6811.881037] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6811.904707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6811.910269] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6813.666884] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 6818.156666] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6820.552816] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6823.446383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6823.586462] Lustre: lustre-OST0000: new disk, initializing [ 6823.589948] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6823.593301] Lustre: 403470:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6823.630422] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6826.458463] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_default_debug -1 all [ 6826.713870] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6826.722759] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6826.797764] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6831.493540] Lustre: DEBUG MARKER: oleg646-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6837.221050] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6837.223126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6837.229601] Lustre: Skipped 1 previous similar message [ 6837.237252] Lustre: Skipped 5 previous similar messages [ 6841.331089] Lustre: server umount lustre-MDT0000 complete [ 6842.338302] LustreError: 401513:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6842.346294] LustreError: 401513:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 6842.954841] LustreError: 401493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788869705 with bad export cookie 1959496425448180573 [ 6842.956936] LustreError: MGC192.168.206.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6842.959152] LustreError: 402309:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 6842.961228] LustreError: 401493:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6843.094256] Lustre: server umount lustre-MDT0001 complete [ 6855.157505] Lustre: server umount lustre-OST0000 complete [ 6867.796818] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing set_hostid [ 6872.008226] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing load_modules_local [ 6884.929684] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 6765 sec ========== 08:15:47 (1788869747) [ 6885.635150] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 08:15:48 (1788869748) === [ 6887.021477] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 08:15:49 (1788869749) === [ 6904.440164] Lustre: DEBUG MARKER: oleg646-server.virtnet: executing unload_modules_local [ 6905.684819] Key type lgssc unregistered [ 6905.849060] LNet: 409822:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6905.854211] LNetError: 409822:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6905.868323] LNet: Removed LNI 192.168.206.146@tcp [ 6906.252551] Key type .llcrypt unregistered [ 6906.255119] Key type ._llcrypt unregistered