[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 480947070 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002344] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.005007] kvm-guest: setup PV IPIs [ 0.008440] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010012] pid_max: default: 32768 minimum: 301 [ 0.011147] LSM: Security Framework initializing [ 0.012067] Yama: becoming mindful. [ 0.013041] SELinux: Initializing. [ 0.015064] *** VALIDATE selinux *** [ 0.023048] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027773] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028155] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031019] *** VALIDATE tmpfs *** [ 0.032458] *** VALIDATE proc *** [ 0.033213] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036157] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038124] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040032] Spectre V2 : User space: Vulnerable [ 0.041009] Speculative Store Bypass: Vulnerable [ 0.044410] debug: unmapping init [mem 0xffffffff9ae59000-0xffffffff9ae60fff] [ 0.046156] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047685] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048022] ... version: 2 [ 0.049009] ... bit width: 48 [ 0.050011] ... generic registers: 4 [ 0.051012] ... value mask: 0000ffffffffffff [ 0.052013] ... max period: 00007fffffffffff [ 0.053012] ... fixed-purpose events: 3 [ 0.054010] ... event mask: 000000070000000f [ 0.055290] rcu: Hierarchical SRCU implementation. [ 0.057394] smp: Bringing up secondary CPUs ... [ 0.058490] x86: Booting SMP configuration: [ 0.059025] .... node #0, CPUs: #1 #2 #3 [ 0.062367] smp: Brought up 1 node, 4 CPUs [ 0.064011] smpboot: Max logical packages: 1 [ 0.065015] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141366] node 0 deferred pages initialised in 73ms [ 0.143161] devtmpfs: initialized [ 0.144214] x86/mm: Memory block size: 128MB [ 0.146893] gcov: version magic: 0x41383552 [ 0.150020] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.153076] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.155281] pinctrl core: initialized pinctrl subsystem [ 0.157216] [ 0.158009] ************************************************************* [ 0.160009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.162012] ** ** [ 0.163008] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.165010] ** ** [ 0.167009] ** This means that this kernel is built to expose internal ** [ 0.169009] ** IOMMU data structures, which may compromise security on ** [ 0.171009] ** your system. ** [ 0.172010] ** ** [ 0.174009] ** If you see this message and you are not debugging the ** [ 0.176013] ** kernel, report this immediately to your vendor! ** [ 0.178009] ** ** [ 0.179009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.181009] ************************************************************* [ 0.183681] NET: Registered protocol family 16 [ 0.184372] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.187064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.189042] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.191360] cpuidle: using governor menu [ 0.192729] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.195412] PCI: Using configuration type 1 for base access [ 0.198120] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.207207] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.208016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.210102] cryptd: max_cpu_qlen set to 1000 [ 0.214288] ACPI: Added _OSI(Module Device) [ 0.216011] ACPI: Added _OSI(Processor Device) [ 0.217048] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.219012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.224437] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.230483] ACPI: Interpreter enabled [ 0.232041] ACPI: PM: (supports S0 S3 S4 S5) [ 0.233025] ACPI: Using IOAPIC for interrupt routing [ 0.234112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.237450] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.248408] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.251038] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.253016] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.255070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.260524] acpiphp: Slot [2] registered [ 0.262122] acpiphp: Slot [5] registered [ 0.264071] acpiphp: Slot [6] registered [ 0.265114] acpiphp: Slot [7] registered [ 0.266108] acpiphp: Slot [8] registered [ 0.267000] acpiphp: Slot [9] registered [ 0.267000] acpiphp: Slot [10] registered [ 0.268125] acpiphp: Slot [3] registered [ 0.269095] acpiphp: Slot [4] registered [ 0.271078] acpiphp: Slot [11] registered [ 0.272096] acpiphp: Slot [12] registered [ 0.273071] acpiphp: Slot [13] registered [ 0.275076] acpiphp: Slot [14] registered [ 0.276089] acpiphp: Slot [15] registered [ 0.277072] acpiphp: Slot [16] registered [ 0.279075] acpiphp: Slot [17] registered [ 0.280090] acpiphp: Slot [18] registered [ 0.281129] acpiphp: Slot [19] registered [ 0.283075] acpiphp: Slot [20] registered [ 0.284090] acpiphp: Slot [21] registered [ 0.285094] acpiphp: Slot [22] registered [ 0.287063] acpiphp: Slot [23] registered [ 0.288104] acpiphp: Slot [24] registered [ 0.289093] acpiphp: Slot [25] registered [ 0.291084] acpiphp: Slot [26] registered [ 0.292083] acpiphp: Slot [27] registered [ 0.294130] acpiphp: Slot [28] registered [ 0.295123] acpiphp: Slot [29] registered [ 0.296099] acpiphp: Slot [30] registered [ 0.298081] acpiphp: Slot [31] registered [ 0.299070] PCI host bridge to bus 0000:00 [ 0.301019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.303020] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.305022] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.308024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.311023] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.314027] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.316170] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.319145] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.323381] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.333014] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.339168] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.342016] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.344018] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.348018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.351512] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.354843] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.358042] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.361993] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.368014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.383024] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.387011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.393438] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.406015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.417017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.449016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.464408] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.480014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.500016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.547019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.562068] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.580016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.595021] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.613024] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.627444] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.640015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.657015] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.708016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.725300] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.733023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.747020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.769026] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.781324] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.803014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.819021] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.838020] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.852180] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.858343] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.862371] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.864484] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.867320] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.873238] iommu: Default domain type: Passthrough [ 0.875634] SCSI subsystem initialized [ 0.877129] ACPI: bus type USB registered [ 0.878084] usbcore: registered new interface driver usbfs [ 0.881071] usbcore: registered new interface driver hub [ 0.883100] usbcore: registered new device driver usb [ 0.885188] pps_core: LinuxPPS API ver. 1 registered [ 0.887012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.890072] PTP clock support registered [ 0.893063] EDAC MC: Ver: 3.0.0 [ 0.895182] PCI: Using ACPI for IRQ routing [ 0.897993] NetLabel: Initializing [ 0.899012] NetLabel: domain hash size = 128 [ 0.901013] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.903136] NetLabel: unlabeled traffic allowed by default [ 0.906237] vgaarb: loaded [ 0.908273] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.910015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.919168] clocksource: Switched to clocksource kvm-clock [ 1.042179] VFS: Disk quotas dquot_6.6.0 [ 1.043954] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.047233] *** VALIDATE ramfs *** [ 1.048758] *** VALIDATE hugetlbfs *** [ 1.051553] pnp: PnP ACPI init [ 1.054306] pnp: PnP ACPI: found 6 devices [ 1.074113] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.078543] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.081329] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.083967] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.086885] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.089793] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.093195] NET: Registered protocol family 2 [ 1.095824] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.101548] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.105690] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.113257] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.117723] TCP: Hash tables configured (established 65536 bind 65536) [ 1.121914] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.126362] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.130544] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.134400] NET: Registered protocol family 1 [ 1.138182] RPC: Registered named UNIX socket transport module. [ 1.141945] RPC: Registered udp transport module. [ 1.144237] RPC: Registered tcp transport module. [ 1.146256] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.149154] NET: Registered protocol family 44 [ 1.150993] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.153522] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.155835] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.158415] PCI: CLS 0 bytes, default 64 [ 1.159834] Unpacking initramfs... [ 2.548656] debug: unmapping init [mem 0xffff99953cc54000-0xffff99953ffbffff] [ 2.555251] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.557851] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.561045] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.094415] Initialise system trusted keyrings [ 3.096365] Key type blacklist registered [ 3.098455] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.107964] zbud: loaded [ 3.111342] *** VALIDATE nfs *** [ 3.112655] *** VALIDATE nfs4 *** [ 3.114654] pstore: using deflate compression [ 3.118647] Platform Keyring initialized [ 3.226621] NET: Registered protocol family 38 [ 3.228915] Key type asymmetric registered [ 3.230439] Asymmetric key parser 'x509' registered [ 3.232464] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.235203] io scheduler mq-deadline registered [ 3.236993] io scheduler kyber registered [ 3.238647] io scheduler bfq registered [ 3.240947] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.244611] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.248598] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.252517] ACPI: Power Button [PWRF] [ 3.259326] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.267201] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.291423] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.303606] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.347836] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.376964] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.429105] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.446907] Non-volatile memory driver v1.3 [ 3.449187] Linux agpgart interface v0.103 [ 3.483733] virtio_blk virtio1: [vda] 149568 512-byte logical blocks (76.6 MB/73.0 MiB) [ 3.487091] vda: detected capacity change from 0 to 76578816 [ 3.512756] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.515726] vdb: detected capacity change from 0 to 1073741824 [ 3.536856] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.540556] vdc: detected capacity change from 0 to 2621440000 [ 3.561280] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.564858] vdd: detected capacity change from 0 to 2621440000 [ 3.585703] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.589181] vde: detected capacity change from 0 to 4294967296 [ 3.606485] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.609441] vdf: detected capacity change from 0 to 4294967296 [ 3.622091] libphy: Fixed MDIO Bus: probed [ 3.627225] usbcore: registered new interface driver usbserial_generic [ 3.629504] usbserial: USB Serial support registered for generic [ 3.632162] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.636734] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.638851] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.641398] mousedev: PS/2 mouse device common for all mice [ 3.644797] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.646608] rtc_cmos 00:05: RTC can wake from S4 [ 3.652544] rtc_cmos 00:05: registered as rtc0 [ 3.655833] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.659485] intel_pstate: CPU model not supported [ 3.662913] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.668267] hid: raw HID events driver (C) Jiri Kosina [ 3.671074] usbcore: registered new interface driver usbhid [ 3.671712] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.673657] usbhid: USB HID core driver [ 3.678524] drop_monitor: Initializing network drop monitor service [ 3.681689] Initializing XFRM netlink socket [ 3.683781] NET: Registered protocol family 10 [ 3.687145] Segment Routing with IPv6 [ 3.688951] NET: Registered protocol family 17 [ 3.690839] mpls_gso: MPLS GSO support [ 3.696989] RAS: Correctable Errors collector initialized. [ 3.698865] AVX version of gcm_enc/dec engaged. [ 3.700670] AES CTR mode by8 optimization enabled [ 3.768563] sched_clock: Marking stable (3768542837, 0)->(4721798514, -953255677) [ 3.771428] registered taskstats version 1 [ 3.773092] Loading compiled-in X.509 certificates [ 3.774931] zswap: loaded using pool lzo/zbud [ 3.799562] Key type big_key registered [ 3.812075] Key type encrypted registered [ 3.813976] ima: No TPM chip found, activating TPM-bypass! [ 3.816498] ima: Allocated hash algorithm: sha1 [ 3.818684] ima: No architecture policies found [ 3.820563] evm: Initialising EVM extended attributes: [ 3.822839] evm: security.selinux [ 3.824347] evm: security.ima [ 3.825693] evm: security.capability [ 3.827304] evm: HMAC attrs: 0x1 [ 3.830269] rtc_cmos 00:05: setting system clock to 2026-08-22 05:44:56 UTC (1787377496) [ 3.837397] debug: unmapping init [mem 0xffffffff9be03000-0xffffffff9bffffff] [ 3.841187] debug: unmapping init [mem 0xffffffff9ab82000-0xffffffff9ae58fff] [ 3.850715] Write protecting the kernel read-only data: 28672k [ 3.853669] debug: unmapping init [mem 0xffffffff99203000-0xffffffff993fffff] [ 3.856298] debug: unmapping init [mem 0xffffffff99b14000-0xffffffff99bfffff] [ 3.889746] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.898833] systemd[1]: Detected virtualization kvm. [ 3.901759] systemd[1]: Detected architecture x86-64. [ 3.903665] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.932905] systemd[1]: No hostname configured. [ 3.935113] systemd[1]: Set hostname to . [ 3.939122] random: systemd: uninitialized urandom read (16 bytes read) [ 3.941909] systemd[1]: Initializing machine ID from random generator. [ 3.983261] random: ln: uninitialized urandom read (6 bytes read) [ 4.067900] random: systemd: uninitialized urandom read (16 bytes read) [ 4.071258] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 4.076745] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.082828] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. Starting Journal Service... [ OK ] Reached target Initrd Root Device. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. Starting Apply Kernel Variables... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Slices. Starting Create Volatile Files and Directories... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.804201] device-mapper: uevent: version 1.0.3 [ 4.806427] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.562212] virtio_net virtio0 ens2: renamed from eth0 [ 5.576480] random: fast init done [ 5.612261] scsi host0: ata_piix [ 5.639057] scsi host1: ata_piix [ 5.640911] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.643940] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.609962] dracut-initqueue[580]: RTNETLINK answers: File exists [ 10.339860] random: crng init done [ 10.341765] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.931899] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.203592] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.468778] SELinux: Disabled at runtime. [ 12.528704] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.539264] systemd[1]: Detected virtualization kvm. [ 12.541896] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.046664] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.050376] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.056924] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.062084] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.067133] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.076684] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.084917] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... Mounting Huge Pages File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice User and Session Slice. Starting udev Coldplug all Devices... [ OK ] Reached target RPC Port Mapper. Mounting POSIX Message Queue File System... [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... [ 13.236265] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. Starting Apply Kernel Variables... [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 13.631229] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.997904] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.087558] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.162840] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.183845] EDAC sbridge: Ver: 1.1.2 [ 15.532531] Key type dns_resolver registered [ 15.871672] NFS: Registering the id_resolver key type [ 15.873635] Key type id_resolver registered [ 15.875159] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg335-server login: [ 48.435996] hrtimer: interrupt took 3988122 ns [ 52.679458] libcfs: loading out-of-tree module taints kernel. [ 52.788201] Key type ._llcrypt registered [ 52.793753] Key type .llcrypt registered [ 52.960076] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 75.699901] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 77.503552] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 77.537922] alg: No test for adler32 (adler32-zlib) [ 79.243674] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 80.097287] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 81.912170] Key type lgssc registered [ 84.074907] Lustre: Echo OBD driver; http://www.lustre.org/ [ 104.287127] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 152.325473] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 168.207630] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 168.266673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 169.659517] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 169.751436] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 169.928574] Lustre: lustre-MDT0000: new disk, initializing [ 170.075599] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 170.099251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 175.354384] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 191.685649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 191.931789] Lustre: 6512:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 192.009470] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 192.023093] Lustre: Skipped 1 previous similar message [ 192.180267] Lustre: lustre-MDT0001: new disk, initializing [ 192.336267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 192.439097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 192.465948] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 198.198833] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 204.455346] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 217.077342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 217.296502] Lustre: lustre-OST0000: new disk, initializing [ 217.303746] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 217.314452] Lustre: 8449:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 217.399446] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 224.240604] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 226.888694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 226.896965] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 226.944255] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 241.685171] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 241.886992] Lustre: lustre-OST0001: new disk, initializing [ 241.892351] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 241.903578] Lustre: 9522:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 241.996766] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 249.034689] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 252.026914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 252.044366] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 252.151775] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 260.467410] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 271.798865] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 278.259697] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing check_logdir /tmp/testlogs/ [ 283.498118] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing yml_node [ 288.307462] Lustre: DEBUG MARKER: Client: 2.17.54.117 [ 290.626634] Lustre: DEBUG MARKER: MDS: 2.17.54.117 [ 293.033420] Lustre: DEBUG MARKER: OSS: 2.17.54.117 [ 294.619605] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 22 01:49:45 EDT 2026 [ 311.631528] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 313.177370] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 329.185403] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 329.195557] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 329.204837] Lustre: Skipped 3 previous similar messages [ 329.219199] Lustre: Skipped 3 previous similar messages [ 334.306131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 334.314650] Lustre: Skipped 3 previous similar messages [ 335.043054] Lustre: server umount lustre-MDT0000 complete [ 338.367963] LustreError: 10398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787377831 with bad export cookie 12144192984107797152 [ 338.373489] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 338.378071] LustreError: 10398:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 339.425878] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 339.428686] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 339.445301] Lustre: Skipped 1 previous similar message [ 339.455570] Lustre: Skipped 1 previous similar message [ 343.475709] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 343.483971] Lustre: Skipped 1 previous similar message [ 344.954611] Lustre: server umount lustre-MDT0001 complete [ 348.551203] Lustre: server umount lustre-OST0000 complete [ 351.911471] Lustre: server umount lustre-OST0001 complete [ 358.190351] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 367.315157] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 410.353891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 410.628804] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 410.655532] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 410.723059] Lustre: lustre-MDT0000: new disk, initializing [ 410.813120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.837955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 414.730688] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 426.643827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 426.773871] Lustre: 15078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 426.863510] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 426.867454] Lustre: Skipped 1 previous similar message [ 426.944950] Lustre: lustre-MDT0001: new disk, initializing [ 427.060305] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 427.109113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 427.124426] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 431.503107] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 440.794706] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 447.287502] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 456.011342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 456.356242] Lustre: lustre-OST0000: new disk, initializing [ 456.359439] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 456.364640] Lustre: 17049:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 456.472767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 458.159733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 458.183216] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 458.266678] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 463.619262] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 474.716429] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 480.884382] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 481.249798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 487.230959] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 487.547791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 494.050466] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 494.054499] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 494.061262] LustreError: Skipped 1 previous similar message [ 494.070612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 494.081100] Lustre: Skipped 1 previous similar message [ 494.106210] Lustre: Skipped 1 previous similar message [ 504.290611] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 504.293243] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 504.310991] Lustre: Skipped 3 previous similar messages [ 504.542548] Lustre: server umount lustre-OST0000 complete [ 508.902557] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 508.911923] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 514.530255] LustreError: 16447:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.547961] LustreError: 16447:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 514.613981] Lustre: server umount lustre-MDT0000 complete [ 517.986420] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378010 with bad export cookie 12144192984107798685 [ 517.997235] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 518.003390] LustreError: 15070:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 518.344915] Lustre: server umount lustre-MDT0001 complete [ 523.602488] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:53:34 (1787378014) [ 530.374168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 530.879372] LustreError: 19317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 530.930945] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 535.167759] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 536.040911] LustreError: 19318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.169263] LustreError: 19317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.550584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 549.077262] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 555.733878] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 561.181234] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 568.012549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 574.077803] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 581.932951] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 585.582268] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 595.938595] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 595.958089] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 595.968471] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 595.968480] Lustre: Skipped 3 previous similar messages [ 596.039399] Lustre: Skipped 2 previous similar messages [ 597.743598] Lustre: server umount lustre-MDT0000 complete [ 601.061622] LustreError: 19318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 601.107858] LustreError: 19318:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 602.284238] LustreError: 20054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378094 with bad export cookie 12144192984107799567 [ 602.300927] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 602.302655] LustreError: 20054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 602.728862] Lustre: server umount lustre-MDT0001 complete [ 617.258149] Lustre: server umount lustre-OST0000 complete [ 640.459835] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 650.454684] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 650.893621] LustreError: 23696:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 650.974381] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 650.983218] Lustre: Skipped 2 previous similar messages [ 655.457398] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 666.247865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 673.241913] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 677.161806] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 685.475832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 692.619824] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 701.242814] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 701.458859] Lustre: lustre-OST0001: new disk, initializing [ 701.464656] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 701.474804] Lustre: 26016:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 702.888860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 702.905260] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 703.012090] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 707.709184] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 717.398159] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 720.930927] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 733.157844] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 733.173418] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 733.196817] Lustre: Skipped 1 previous similar message [ 733.204414] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 733.212787] Lustre: Skipped 2 previous similar messages [ 739.607250] Lustre: server umount lustre-MDT0000 complete [ 743.675526] LustreError: 23676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378236 with bad export cookie 12144192984107800407 [ 743.684303] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 743.930522] LustreError: 25485:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 743.947165] LustreError: 25485:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 744.202350] Lustre: server umount lustre-MDT0001 complete [ 750.282588] Lustre: server umount lustre-OST0000 complete [ 754.457750] Lustre: server umount lustre-OST0001 complete [ 770.335719] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 781.058967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 781.482115] LustreError: 29031:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 781.565419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 781.585161] Lustre: Skipped 3 previous similar messages [ 786.623426] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 795.613081] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 801.305089] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 804.720574] Lustre: 30179:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 813.259667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 821.834826] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 830.949285] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 838.660087] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 847.228965] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 851.064718] Lustre: 32060:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 857.570586] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 857.589684] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 857.613371] Lustre: Skipped 5 previous similar messages [ 857.619177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 857.627470] Lustre: Skipped 7 previous similar messages [ 863.832333] Lustre: server umount lustre-MDT0000 complete [ 867.308732] LustreError: 29026:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 867.334157] LustreError: 29026:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 867.771541] LustreError: 29011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378360 with bad export cookie 12144192984107801702 [ 867.774938] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 867.780770] LustreError: 29011:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 868.154941] Lustre: server umount lustre-MDT0001 complete [ 883.159847] Lustre: server umount lustre-OST0000 complete [ 897.547589] Lustre: server umount lustre-OST0001 complete [ 907.270064] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:59:57 (1787378397) [ 940.548893] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 951.707378] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 957.266772] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 966.821534] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 972.788696] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 976.270585] Lustre: 36028:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 985.032439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 992.525423] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 996.800820] LustreError: 36695:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 996.829683] LustreError: 36695:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 1001.969386] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1009.280925] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1017.490068] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1021.396470] Lustre: 37910:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1028.068906] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1028.091366] Lustre: Skipped 5 previous similar messages [ 1028.097431] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1028.118395] Lustre: Skipped 7 previous similar messages [ 1033.381224] Lustre: server umount lustre-MDT0000 complete [ 1037.132174] LustreError: 34846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378529 with bad export cookie 12144192984107802815 [ 1037.136990] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1037.141927] LustreError: 34846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1037.433697] Lustre: server umount lustre-MDT0001 complete [ 1050.567491] Lustre: server umount lustre-OST0000 complete [ 1053.667021] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787378530/real 1787378530] req@ffff99959ead8e00 x1874201225283968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787378546 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1054.969910] Lustre: server umount lustre-OST0001 complete [ 1064.077407] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 02:02:34 (1787378554) [ 1081.238689] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1090.944540] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1091.366502] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1091.374984] Lustre: Skipped 7 previous similar messages [ 1095.924963] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1104.375858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1109.754560] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1112.633490] Lustre: 41200:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1119.578509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1125.571827] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1134.549362] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1141.926140] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1149.697861] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1153.235764] Lustre: 43078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1154.164444] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1154.169619] Lustre: Skipped 1 previous similar message [ 1161.185768] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1161.194583] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1161.212518] Lustre: Skipped 2 previous similar messages [ 1163.967818] Lustre: server umount lustre-MDT0000 complete [ 1167.477738] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378660 with bad export cookie 12144192984107803921 [ 1167.486361] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1167.492671] LustreError: 40030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1173.878518] Lustre: server umount lustre-MDT0001 complete [ 1182.953083] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1197.929767] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1203.190809] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1207.742265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1213.399642] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1213.444868] Lustre: lustre-MDT0001-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1213.453656] Lustre: Skipped 1 previous similar message [ 1221.319556] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1223.470603] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1229.364490] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1231.286647] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1239.009982] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1242.587723] Lustre: server umount lustre-MDT0000 complete [ 1246.593960] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378739 with bad export cookie 12144192984107805020 [ 1246.596758] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1246.607394] LustreError: 40032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1247.136120] Lustre: server umount lustre-MDT0001 complete [ 1261.625443] Lustre: server umount lustre-OST0000 complete [ 1264.609066] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787378741/real 1787378741] req@ffff9995bd8e5880 x1874201225369728/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787378757 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1265.104568] Lustre: server umount lustre-OST0001 complete [ 1274.693859] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 02:06:05 (1787378765) [ 1290.684891] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1299.991899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1300.348468] LustreError: 47685:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1300.372864] LustreError: 47685:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 29 previous similar messages [ 1304.615386] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1312.921350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1318.466162] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1321.431550] Lustre: 48840:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1328.744107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1336.208868] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1344.916070] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1352.158978] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1360.217224] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1365.374221] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1377.263777] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1377.279249] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1377.281134] Lustre: Skipped 13 previous similar messages [ 1377.297852] Lustre: Skipped 21 previous similar messages [ 1377.549428] Lustre: server umount lustre-MDT0000 complete [ 1381.558799] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787378874 with bad export cookie 12144192984107805839 [ 1381.571629] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1381.579439] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1381.865111] Lustre: server umount lustre-MDT0001 complete [ 1396.437954] Lustre: server umount lustre-OST0000 complete [ 1399.334563] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787378875/real 1787378875] req@ffff9995b49f9f80 x1874201225424896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787378891 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1400.527820] Lustre: server umount lustre-OST0001 complete [ 1410.515740] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 02:08:21 (1787378901) [ 1412.658101] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1414.477703] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 02:08:25 (1787378905) [ 1425.352293] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 1433.689749] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1457.271819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1457.561038] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1457.652565] Lustre: lustre-MDT0000: new disk, initializing [ 1457.733271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1462.132724] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1472.183354] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1479.605140] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1480.041387] Lustre: lustre-OST0000: new disk, initializing [ 1480.046518] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1480.052353] Lustre: Skipped 1 previous similar message [ 1480.061594] Lustre: 54916:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1482.088158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1482.105163] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1482.142884] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1487.874241] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1497.447297] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1499.596284] Lustre: DEBUG MARKER: create test files [ 1533.412387] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1537.617211] Lustre: server umount lustre-OST0000 complete [ 1542.577600] Lustre: server umount lustre-MDT0000 complete [ 1555.318702] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1557.798706] Key type lgssc unregistered [ 1558.103456] LNet: 56546:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1558.111162] LNetError: 56546:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1558.130919] LNet: Removed LNI 192.168.203.135@tcp [ 1559.006768] Key type .llcrypt unregistered [ 1559.013948] Key type ._llcrypt unregistered [ 1590.158403] Key type ._llcrypt registered [ 1590.170830] Key type .llcrypt registered [ 1590.340568] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1592.079217] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1592.118872] alg: No test for adler32 (adler32-zlib) [ 1593.246806] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 1593.470560] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1595.114183] Key type lgssc registered [ 1596.295224] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1604.314668] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1604.357679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1605.854330] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1610.253666] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1618.069821] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1624.781553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1625.225257] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1630.421105] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1630.569479] Lustre: 59282:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1630.599933] Lustre: 59282:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1630.617576] Lustre: 59282:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1630.630791] Lustre: 59282:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1630.644888] Lustre: 59282:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1630.657375] Lustre: 59282:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1632.702467] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1641.783097] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1648.123136] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1653.420694] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1661.413478] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1661.423324] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1661.455205] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1662.971701] Lustre: server umount lustre-OST0000 complete [ 1666.978319] Lustre: server umount lustre-MDT0000 complete [ 1679.186612] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1682.754153] Key type lgssc unregistered [ 1683.139327] LNet: 60889:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1683.149310] LNetError: 60889:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1683.191880] LNet: Removed LNI 192.168.203.135@tcp [ 1684.306959] Key type .llcrypt unregistered [ 1684.308744] Key type ._llcrypt unregistered [ 1713.251157] Key type ._llcrypt registered [ 1713.253301] Key type .llcrypt registered [ 1713.348597] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1714.445162] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1714.465809] alg: No test for adler32 (adler32-zlib) [ 1715.567552] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 1715.907440] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1717.600568] Key type lgssc registered [ 1718.801689] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1726.665440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1726.721553] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1728.387233] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1733.099509] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1739.965980] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1746.730377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1747.103620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1752.965930] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1754.300102] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1754.398586] Lustre: 63703:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1754.406102] Lustre: 63703:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1754.412827] Lustre: 63703:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1754.426944] Lustre: 63703:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1754.441653] Lustre: 63703:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1754.450769] Lustre: 63703:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1760.666759] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1765.393706] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1775.093468] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1775.100303] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1775.118603] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1780.047874] Lustre: server umount lustre-OST0000 complete [ 1784.216449] Lustre: server umount lustre-MDT0000 complete [ 1796.592942] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 1799.900296] Key type lgssc unregistered [ 1800.248037] LNet: 65261:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1800.253988] LNetError: 65261:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1800.290970] LNet: Removed LNI 192.168.203.135@tcp [ 1801.201214] Key type .llcrypt unregistered [ 1801.207153] Key type ._llcrypt unregistered [ 1817.808926] Key type ._llcrypt registered [ 1817.813729] Key type .llcrypt registered [ 1817.921604] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 1832.331476] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 1833.205875] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1833.243430] alg: No test for adler32 (adler32-zlib) [ 1834.346049] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 1834.558383] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 1836.232147] Key type lgssc registered [ 1837.298813] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1871.180608] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1871.201364] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1872.560553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1872.592705] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1872.656125] Lustre: lustre-MDT0000: new disk, initializing [ 1872.715321] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1872.730090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1877.154963] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1887.568557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1887.682740] Lustre: 68415:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1887.729970] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1887.733988] Lustre: Skipped 1 previous similar message [ 1887.793334] Lustre: lustre-MDT0001: new disk, initializing [ 1887.892114] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1887.919570] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1887.936080] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1892.013621] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1900.945700] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1906.330408] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1913.892680] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1914.203609] Lustre: lustre-OST0000: new disk, initializing [ 1914.206969] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1914.211441] Lustre: 70382:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1914.293341] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1919.567701] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 1920.585947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1920.597541] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1920.694616] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1928.517858] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1932.317759] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1932.547288] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1936.503495] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1936.682713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1941.475166] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1941.483875] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1941.486435] LustreError: Skipped 1 previous similar message [ 1941.499691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1941.502593] Lustre: Skipped 1 previous similar message [ 1941.531423] Lustre: Skipped 1 previous similar message [ 1944.768402] Lustre: server umount lustre-OST0000 complete [ 1949.668568] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1949.685270] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1949.704733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1951.716098] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1951.716521] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1951.730469] Lustre: Skipped 1 previous similar message [ 1954.206256] Lustre: server umount lustre-MDT0000 complete [ 1958.127658] LustreError: 68407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787379450 with bad export cookie 10509800516837764124 [ 1958.137953] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1958.143796] LustreError: 68407:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1958.603335] Lustre: server umount lustre-MDT0001 complete [ 1968.999912] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:17:39 (1787379459) [ 1980.469606] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 2021.989678] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 2031.828566] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 2067.059761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2067.391623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2067.416621] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2067.492879] Lustre: lustre-MDT0000: new disk, initializing [ 2067.593741] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2067.614431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2071.267913] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2081.341290] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2081.487551] Lustre: 76401:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2081.565265] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2081.570411] Lustre: Skipped 1 previous similar message [ 2081.672270] Lustre: lustre-MDT0001: new disk, initializing [ 2081.798120] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2081.891403] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2081.911518] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2086.454639] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2095.589452] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2101.410751] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2108.443454] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2108.887885] Lustre: lustre-OST0000: new disk, initializing [ 2108.892678] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2108.898932] Lustre: 78371:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2109.015047] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2116.207912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2116.223070] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2116.351058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2116.772597] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2128.091552] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2133.424876] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2133.748494] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2138.982303] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2139.219469] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2142.177973] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2142.181837] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2142.190934] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2142.216720] Lustre: Skipped 1 previous similar message [ 2142.241961] Lustre: Skipped 1 previous similar message [ 2147.302269] LustreError: 78354:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2147.331217] LustreError: 78354:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2147.457472] Lustre: server umount lustre-OST0000 complete [ 2152.418419] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2152.422751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2152.438993] Lustre: Skipped 1 previous similar message [ 2157.847610] Lustre: server umount lustre-MDT0000 complete [ 2161.350939] LustreError: 78381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787379654 with bad export cookie 10509800516837765006 [ 2161.354270] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2161.364263] LustreError: 78381:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2161.805960] Lustre: server umount lustre-MDT0001 complete [ 2169.987843] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:21:00 (1787379660) [ 2176.419040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2176.818931] LustreError: 80783:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2176.877225] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2181.208907] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2182.118066] LustreError: 80784:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2187.242655] LustreError: 80783:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2189.677806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2195.055608] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2202.933229] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2209.034521] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2216.253804] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2216.630233] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2216.638929] Lustre: Skipped 1 previous similar message [ 2222.042860] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2228.344609] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2231.406838] Lustre: 83454:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting f247ff3f-a3cb-4b80-8671-8e6c9b481989 at adminstrative request [ 2232.295762] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2232.309374] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2241.578348] Lustre: MGS: Client f247ff3f-a3cb-4b80-8671-8e6c9b481989 (at 0@lo) reconnecting [ 2241.583324] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x91da4fe30254f2f9 to 0x91da4fe30254f599 [ 2241.590225] Lustre: MGC192.168.203.135@tcp: Connection restored to 0@lo (at 0@lo) [ 2267.919076] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2268.098201] Lustre: lustre-OST0001: new disk, initializing [ 2268.103541] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2268.113169] Lustre: 83850:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2273.684966] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2276.439168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2276.452720] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2276.568942] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2282.322881] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2287.078489] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2287.087843] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2287.119852] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2287.128745] Lustre: Skipped 4 previous similar messages [ 2299.360150] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2299.489895] Lustre: server umount lustre-OST0001 complete [ 2302.436732] LustreError: 83467:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2302.453085] LustreError: 83467:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2303.969533] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2303.979957] LustreError: Skipped 1 previous similar message [ 2303.983276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2303.995251] Lustre: Skipped 1 previous similar message [ 2304.001013] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2304.008228] Lustre: Skipped 4 previous similar messages [ 2312.680737] LustreError: 82604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2312.700659] LustreError: 82604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2317.280170] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2317.438537] Lustre: server umount lustre-OST0000 complete [ 2327.516209] Lustre: server umount lustre-MDT0000 complete [ 2330.651405] LustreError: 80763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787379823 with bad export cookie 10509800516837766553 [ 2330.661761] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2330.665614] LustreError: 80763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2330.919596] Lustre: server umount lustre-MDT0001 complete [ 2339.708843] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:23:50 (1787379830) [ 2341.351708] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2343.080219] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:23:53 (1787379833) [ 2344.596416] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2346.081273] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:23:57 (1787379837) [ 2347.539822] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2349.057775] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:24:00 (1787379840) [ 2350.786074] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2352.583716] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:24:03 (1787379843) [ 2354.066378] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2359.747073] loop: module loaded [ 2359.778761] loop0: detected capacity change from 0 to 4294967296 [ 2359.833512] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2361.455229] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2364.662241] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2368.329978] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2396.776916] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2396.823363] LustreError: 87975:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2396.838124] LustreError: 87975:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2396.854706] LustreError: 87975:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2396.872678] LustreError: 87975:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2416.800154] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 2425.551195] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 2465.330836] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2465.579869] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2465.612871] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2465.696439] Lustre: lustre-MDT0000: new disk, initializing [ 2465.797966] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2465.811877] Lustre: Skipped 1 previous similar message [ 2465.823489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2471.028686] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2481.939897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2482.078573] Lustre: 90850:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2482.368853] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2486.890466] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2497.910458] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2505.900079] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2513.940089] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2514.233919] Lustre: 92820:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2520.860953] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2524.360546] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2524.522457] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2531.316703] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2536.390265] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2536.835887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2541.588189] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2541.871937] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2545.121808] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2545.125210] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2545.132329] LustreError: Skipped 1 previous similar message [ 2545.134642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2545.148982] Lustre: Skipped 4 previous similar messages [ 2545.154186] Lustre: Skipped 7 previous similar messages [ 2550.525755] Lustre: server umount lustre-OST0000 complete [ 2560.484764] LustreError: 92015:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2560.517966] LustreError: 92015:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2560.630173] Lustre: server umount lustre-MDT0000 complete [ 2564.428892] LustreError: 90844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380057 with bad export cookie 10509800516837767176 [ 2564.440335] LustreError: 90844:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2564.440864] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2564.844955] Lustre: server umount lustre-MDT0001 complete [ 2573.331323] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:27:43 (1787380063) [ 2580.405165] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2585.433993] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2593.638336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2593.982222] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2593.991217] Lustre: Skipped 3 previous similar messages [ 2598.198277] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2604.919510] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2610.271213] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2617.085995] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2622.815238] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2630.736335] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2635.451330] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2635.630867] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2639.970272] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2640.182908] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2646.770568] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2646.964280] Lustre: lustre-OST0001: new disk, initializing [ 2646.967692] Lustre: Skipped 2 previous similar messages [ 2646.972659] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2646.979877] Lustre: Skipped 3 previous similar messages [ 2646.988920] Lustre: 98612:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2652.802728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2652.818545] Lustre: Skipped 2 previous similar messages [ 2652.823220] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2652.979074] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2653.085766] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2661.868679] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2665.589638] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2665.782616] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2669.495330] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2669.727917] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2688.161473] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2715.500394] Lustre: Failing over lustre-MDT0000 [ 2716.196281] Lustre: server umount lustre-MDT0000 complete [ 2716.642409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2716.655449] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2716.687685] Lustre: Skipped 2 previous similar messages [ 2716.703327] LustreError: 95245:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2716.739711] LustreError: 95245:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2736.098924] Lustre: 66636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787380212/real 1787380212] req@00000000a3d8b573 x1874203065463936/t0(0) o400->MGC192.168.203.135@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787380228 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2736.144710] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2744.824433] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2744.827676] LDISKFS-fs (dm-0): recovery complete [ 2744.833358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2746.016181] LustreError: 101430:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 2746.034672] LustreError: 101430:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@000000001ccb8d1a x1874203065469312/t0(0) o250->MGC192.168.203.135@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787380238 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2746.071418] LustreError: 101430:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 2746.748351] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2748.884169] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2748.885592] Lustre: 95240:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2751.364928] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2751.979081] Lustre: 95244:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2751.981977] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2751.995812] Lustre: 95244:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 2752.125359] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2752.449689] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2752.797723] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2752.810146] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2753.154085] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2753.158403] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 2753.164501] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2753.177670] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 2753.826828] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2753.842188] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2754.168133] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2754.177963] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 2755.474935] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2755.488340] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 6 previous similar messages [ 2755.827782] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2755.844344] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 2756.488642] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2756.498133] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 6 previous similar messages [ 2759.673054] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2759.687744] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 2760.018445] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2760.032580] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2760.680377] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2760.690115] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 2767.978840] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2767.997409] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 25 previous similar messages [ 2768.328844] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2768.341492] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2768.976170] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2768.982207] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 25 previous similar messages [ 2784.261063] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2784.281116] Lustre: 101461:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 50 previous similar messages [ 2784.601183] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2784.624609] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2785.288335] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2785.303967] LustreError: 101461:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 50 previous similar messages [ 2808.216154] Lustre: 101461:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2808.229333] Lustre: 101461:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2808.244951] Lustre: 101461:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ef0cc99e-115d-40cc-929f-793f4a4fcd00@192.168.203.35@tcp [ 2808.260684] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2808.281476] Lustre: 101461:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@0000000039b3784e x1874203050576512/t0(8589934772) o35->ef0cc99e-115d-40cc-929f-793f4a4fcd00@192.168.203.35@tcp:106/0 lens 392/0 e 0 to 0 dl 1787380311 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2808.328168] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2808.349311] Lustre: lustre-MDT0000: Denying connection for new client ef0cc99e-115d-40cc-929f-793f4a4fcd00 (at 192.168.203.35@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2808.363067] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2808.443027] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2808.463613] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2808.465288] Lustre: Skipped 2 previous similar messages [ 2808.538411] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 2808.542178] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2823.663605] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2823.666691] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2823.687439] Lustre: Skipped 6 previous similar messages [ 2827.977867] Lustre: server umount lustre-OST0000 complete [ 2828.775513] LustreError: 97065:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2828.785244] LustreError: 97065:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 43 previous similar messages [ 2838.645142] Lustre: server umount lustre-OST0001 complete [ 2849.592617] Lustre: server umount lustre-MDT0000 complete [ 2853.560201] LustreError: 95226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380346 with bad export cookie 10509800516837818388 [ 2853.563927] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2853.573265] LustreError: 95226:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2854.069444] Lustre: server umount lustre-MDT0001 complete [ 2863.673393] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:32:34 (1787380354) [ 2872.638553] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2872.655624] Lustre: *** cfs_fail_loc=197, val=0*** [ 2888.544800] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2893.664554] LustreError: 103436:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.135@tcp: failed processing log, type 4: rc = -110 [ 2898.797477] Lustre: *** cfs_fail_loc=197, val=0*** [ 2898.800157] Lustre: Skipped 65 previous similar messages [ 2919.328395] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2919.343484] Lustre: Skipped 3 previous similar messages [ 2926.993940] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2935.505819] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2937.422443] Lustre: server umount lustre-OST0000 complete [ 2945.665611] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:33:56 (1787380436) [ 2967.252593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2967.760267] LustreError: 105059:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2967.776315] LustreError: 105059:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 2972.144892] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 2981.945357] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2985.773638] LustreError: 105090:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2985.789216] LustreError: 105090:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2985.798839] LustreError: 105090:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 2986.186623] Lustre: server umount lustre-MDT0000 complete [ 3018.358438] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 3028.119540] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3062.894424] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3063.090279] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3063.152810] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3063.276664] Lustre: lustre-MDT0000: new disk, initializing [ 3063.387903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3067.844029] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3078.769249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3078.912489] Lustre: 109262:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3079.348703] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3084.372047] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3094.670792] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3100.445200] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3107.691465] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3108.220930] Lustre: 111232:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3115.207286] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3116.757085] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3126.070741] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3132.369352] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3132.599794] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3137.933583] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3138.166150] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3142.625179] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3142.629498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3142.660914] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3142.683182] Lustre: Skipped 10 previous similar messages [ 3142.696468] Lustre: Skipped 6 previous similar messages [ 3147.057609] Lustre: server umount lustre-OST0000 complete [ 3157.621300] Lustre: server umount lustre-MDT0000 complete [ 3162.227164] LustreError: 109254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380654 with bad export cookie 10509800516837829000 [ 3162.232273] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3162.244175] LustreError: 109254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3162.660490] Lustre: server umount lustre-MDT0001 complete [ 3172.251554] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:37:42 (1787380662) [ 3181.886896] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3185.137157] Key type lgssc unregistered [ 3185.505577] LNet: 113720:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3185.517778] LNetError: 113720:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3186.539159] LNet: Removed LNI 192.168.203.135@tcp [ 3187.414686] Key type .llcrypt unregistered [ 3187.417119] Key type ._llcrypt unregistered [ 3205.763160] Key type ._llcrypt registered [ 3205.765474] Key type .llcrypt registered [ 3205.897910] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 3223.627540] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3225.283744] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3225.561530] alg: No test for adler32 (adler32-zlib) [ 3226.747837] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 3227.181295] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 3228.924143] Key type lgssc registered [ 3230.741926] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3272.923943] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3273.002144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3274.320305] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3274.368958] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3274.476337] Lustre: lustre-MDT0000: new disk, initializing [ 3274.536493] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3274.568423] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3279.184202] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3290.110528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3290.321202] Lustre: lustre-OST0000: new disk, initializing [ 3290.324493] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3290.327555] Lustre: Skipped 1 previous similar message [ 3290.331813] Lustre: 117905:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3290.382544] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3291.565492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3291.575648] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3291.624196] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3297.756852] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3308.536277] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3308.707595] Lustre: lustre-OST0001: new disk, initializing [ 3308.713678] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3308.722139] Lustre: 118768:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3308.805992] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3314.828633] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3314.845806] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3314.875994] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3315.353733] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3325.421147] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3325.427146] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3325.439200] Lustre: Skipped 1 previous similar message [ 3325.448854] Lustre: Skipped 1 previous similar message [ 3329.603667] Lustre: server umount lustre-MDT0000 complete [ 3335.310286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3339.039392] LustreError: 117091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787380831 with bad export cookie 2015080721513361663 [ 3339.044220] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3339.051857] LustreError: 117091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3339.110118] Lustre: server umount lustre-OST0000 complete [ 3342.956383] Lustre: server umount lustre-OST0001 complete [ 3350.855928] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 02:40:41 (1787380841) [ 3374.008478] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 02:41:04 (1787380864) [ 3384.436508] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3407.834806] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:41:38 (1787380898) [ 3451.524903] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 3463.139857] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3507.704279] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3508.117127] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3508.204317] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3508.382850] Lustre: lustre-MDT0000: new disk, initializing [ 3508.604080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3508.663494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3514.518572] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3526.424574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3526.593685] Lustre: 126530:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3526.666581] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3526.674935] Lustre: Skipped 1 previous similar message [ 3526.780634] Lustre: lustre-MDT0001: new disk, initializing [ 3526.917886] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3526.958853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3526.989719] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3532.244215] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3541.735517] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3547.461581] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3554.748393] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3555.030405] Lustre: lustre-OST0000: new disk, initializing [ 3555.033616] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3555.039202] Lustre: 128499:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3555.107035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3561.160045] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3563.586114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3563.595326] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3563.696251] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3572.040849] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3577.211331] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3577.479558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3583.131744] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3583.412031] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3589.602944] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3589.603611] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3589.616216] LustreError: Skipped 1 previous similar message [ 3589.617622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3589.656300] Lustre: Skipped 1 previous similar message [ 3591.716283] Lustre: server umount lustre-OST0000 complete [ 3598.817220] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3598.825827] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3598.839961] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3598.846953] Lustre: Skipped 1 previous similar message [ 3601.462656] Lustre: server umount lustre-MDT0000 complete [ 3604.631225] LustreError: 127324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381097 with bad export cookie 2015080721513362888 [ 3604.637287] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3604.658064] LustreError: 127324:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3604.930326] Lustre: server umount lustre-MDT0001 complete [ 3613.216598] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:45:04 (1787381104) [ 3619.972556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3620.475978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3624.714472] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3625.964585] LustreError: 130912:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3625.989296] LustreError: 130912:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3631.073980] LustreError: 130913:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3632.963758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3637.150383] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3644.836413] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3650.390860] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3658.367546] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3658.887776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3658.903395] Lustre: Skipped 1 previous similar message [ 3665.364337] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3673.055652] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3681.307613] Lustre: *** cfs_fail_loc=159, val=0*** [ 3697.718706] Lustre: lustre-MDT0000: Client 8cacda02-192b-4b79-900a-e8482f7de6de (at 192.168.203.35@tcp) reconnecting [ 3699.066976] Lustre: *** cfs_fail_loc=159, val=0*** [ 3699.075782] Lustre: Skipped 5 previous similar messages [ 3715.590994] Lustre: lustre-MDT0000: Client 8cacda02-192b-4b79-900a-e8482f7de6de (at 192.168.203.35@tcp) reconnecting [ 3732.964741] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3732.972853] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3732.984868] Lustre: Skipped 1 previous similar message [ 3732.992336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3732.999938] Lustre: Skipped 2 previous similar messages [ 3735.009197] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3738.488504] Lustre: server umount lustre-OST0000 complete [ 3745.251945] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3745.266422] Lustre: Skipped 1 previous similar message [ 3745.276438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3745.282700] Lustre: Skipped 3 previous similar messages [ 3748.345204] Lustre: server umount lustre-MDT0000 complete [ 3750.371701] LustreError: 130913:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3750.407589] LustreError: 130913:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3752.136879] LustreError: 130892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381244 with bad export cookie 2015080721513363770 [ 3752.146991] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3752.149671] LustreError: 130892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3752.530832] Lustre: server umount lustre-MDT0001 complete [ 3763.917611] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 3766.798945] Key type lgssc unregistered [ 3767.108921] LNet: 134690:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3767.120754] LNetError: 134690:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3767.174713] LNet: Removed LNI 192.168.203.135@tcp [ 3768.012321] Key type .llcrypt unregistered [ 3768.015957] Key type ._llcrypt unregistered [ 3791.870906] Key type ._llcrypt registered [ 3791.873436] Key type .llcrypt registered [ 3791.972716] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 3793.058986] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3793.072669] alg: No test for adler32 (adler32-zlib) [ 3794.139308] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 3794.380504] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 3796.032163] Key type lgssc registered [ 3797.316404] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3806.362736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3806.385619] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3807.843200] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3812.675099] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3817.454791] LustreError: 136104:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3817.500126] LustreError: 136104:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3822.237684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3822.662187] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3828.313586] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3836.352263] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3842.931349] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3849.871273] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3850.260862] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3853.361531] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3857.070291] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 3865.772960] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3876.202738] Lustre: *** cfs_fail_loc=159, val=0*** [ 3892.734112] Lustre: lustre-MDT0000: Client 536c2a2b-fc62-4d47-8ec8-a56dfce4fbf0 (at 192.168.203.35@tcp) reconnecting [ 3895.533126] Lustre: *** cfs_fail_loc=159, val=0*** [ 3911.165791] Lustre: lustre-MDT0001: Client 536c2a2b-fc62-4d47-8ec8-a56dfce4fbf0 (at 192.168.203.35@tcp) reconnecting [ 3912.679808] Lustre: *** cfs_fail_loc=159, val=0*** [ 3912.688083] Lustre: Skipped 4 previous similar messages [ 3929.081899] Lustre: lustre-MDT0001: Client 536c2a2b-fc62-4d47-8ec8-a56dfce4fbf0 (at 192.168.203.35@tcp) reconnecting [ 3934.054253] Lustre: 139143:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3937.412959] Lustre: *** cfs_fail_loc=159, val=0*** [ 3937.420807] Lustre: Skipped 3 previous similar messages [ 3953.147286] Lustre: lustre-MDT0000: Client 33136147-8d82-42d6-9661-83d7eb3e33df (at 192.168.203.35@tcp) reconnecting [ 3954.967624] Lustre: *** cfs_fail_loc=159, val=0*** [ 3954.970073] Lustre: Skipped 16 previous similar messages [ 3971.620808] Lustre: lustre-MDT0000: Client 33136147-8d82-42d6-9661-83d7eb3e33df (at 192.168.203.35@tcp) reconnecting [ 3973.390544] Lustre: 139431:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3979.752853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3979.757328] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3979.761785] LustreError: Skipped 1 previous similar message [ 3979.790562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3980.272924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3985.151057] Lustre: server umount lustre-OST0000 complete [ 3990.502010] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3990.508931] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3990.526480] Lustre: Skipped 1 previous similar message [ 3990.551505] Lustre: Skipped 3 previous similar messages [ 3995.448529] Lustre: server umount lustre-MDT0000 complete [ 3995.639659] LustreError: 136099:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3995.672521] LustreError: 136099:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3999.948528] LustreError: 136086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381492 with bad export cookie 1649608035402875929 [ 3999.949547] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3999.960184] LustreError: 136086:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4000.411414] Lustre: server umount lustre-MDT0001 complete [ 4013.055208] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4016.050501] Key type lgssc unregistered [ 4016.368566] LNet: 140484:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4016.382230] LNetError: 140484:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4016.407320] LNet: Removed LNI 192.168.203.135@tcp [ 4017.312713] Key type .llcrypt unregistered [ 4017.317576] Key type ._llcrypt unregistered [ 4041.729386] Key type ._llcrypt registered [ 4041.731433] Key type .llcrypt registered [ 4041.801746] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4042.764641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4042.792886] alg: No test for adler32 (adler32-zlib) [ 4043.893938] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4044.117326] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4045.800378] Key type lgssc registered [ 4047.512931] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4057.110818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4057.153855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4058.776786] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4064.114887] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4069.352440] LustreError: 141902:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4069.379487] LustreError: 141902:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4073.849828] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4074.325760] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4079.419996] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4087.471098] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4093.135522] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4099.430671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4099.906308] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4105.018549] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4105.927795] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4113.424731] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4119.398765] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4123.382587] Lustre: 144485:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4126.648879] Lustre: 144533:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4128.295957] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4129.243879] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4132.990532] Lustre: 144680:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4134.077822] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4135.002550] Lustre: 144790:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4140.004822] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4140.011695] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4140.022895] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4140.039612] Lustre: Skipped 1 previous similar message [ 4145.123910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4145.140190] Lustre: Skipped 1 previous similar message [ 4145.723438] Lustre: server umount lustre-OST0000 complete [ 4150.243362] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4150.261314] Lustre: Skipped 1 previous similar message [ 4150.268341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4150.274280] Lustre: Skipped 1 previous similar message [ 4155.362148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4155.373124] Lustre: Skipped 1 previous similar message [ 4156.024137] Lustre: server umount lustre-MDT0000 complete [ 4159.719869] LustreError: 144741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381652 with bad export cookie 7219778281488025587 [ 4159.727366] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4159.733971] LustreError: 144741:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4160.122271] Lustre: server umount lustre-MDT0001 complete [ 4170.289621] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4172.705308] Key type lgssc unregistered [ 4172.946727] LNet: 145846:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4172.952769] LNetError: 145846:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4172.972286] LNet: Removed LNI 192.168.203.135@tcp [ 4173.642548] Key type .llcrypt unregistered [ 4173.653154] Key type ._llcrypt unregistered [ 4197.929865] Key type ._llcrypt registered [ 4197.932280] Key type .llcrypt registered [ 4198.020665] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4199.354318] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4199.390199] alg: No test for adler32 (adler32-zlib) [ 4200.491144] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4200.706296] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4202.352266] Key type lgssc registered [ 4203.821241] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4212.082481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4212.113073] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4213.876513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4219.463753] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4223.464855] LustreError: 147259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4223.482817] LustreError: 147259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4228.579079] LustreError: 147259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4230.528141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4231.025591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4238.035742] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4248.155652] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4254.549303] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4262.137643] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4262.684761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4269.114232] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4275.966803] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4277.526627] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4285.164903] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4293.669644] Lustre: *** cfs_fail_loc=159, val=0*** [ 4293.686909] Lustre: Skipped 1 previous similar message [ 4308.991931] Lustre: lustre-MDT0000: Client fe172e27-987a-48bb-af75-0c0a0aa8e574 (at 192.168.203.35@tcp) reconnecting [ 4315.112464] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4315.124487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4315.125459] LustreError: Skipped 1 previous similar message [ 4315.167614] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4316.016783] Lustre: server umount lustre-OST0000 complete [ 4321.250066] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4321.261835] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4321.269866] Lustre: Skipped 1 previous similar message [ 4321.291972] Lustre: Skipped 1 previous similar message [ 4325.862930] Lustre: server umount lustre-MDT0000 complete [ 4326.370054] LustreError: 147259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4326.388939] LustreError: 147259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4329.591794] LustreError: 147241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787381822 with bad export cookie 10574497342915427511 [ 4329.598120] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4329.605856] LustreError: 147241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4340.405968] Lustre: server umount lustre-MDT0001 complete [ 4352.462460] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4355.158796] Key type lgssc unregistered [ 4355.430506] LNet: 150945:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4355.439887] LNetError: 150945:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4355.464195] LNet: Removed LNI 192.168.203.135@tcp [ 4356.308596] Key type .llcrypt unregistered [ 4356.311789] Key type ._llcrypt unregistered [ 4381.608388] Key type ._llcrypt registered [ 4381.615164] Key type .llcrypt registered [ 4381.741688] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4382.625106] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4382.671144] alg: No test for adler32 (adler32-zlib) [ 4383.703879] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4383.893771] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4385.585105] Key type lgssc registered [ 4386.707228] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4394.402863] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4394.417925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4395.903254] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4400.036829] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4406.243893] LustreError: 152362:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4406.271215] LustreError: 152362:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4408.347834] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4408.645263] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4413.428766] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4421.171566] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4427.436365] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4434.256683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4434.346177] LustreError: 154189:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4434.680768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4440.048186] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4441.105294] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4441.255762] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4449.152090] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4469.552321] Lustre: lustre-OST0000: evicting b9082a6e-c00f-4d09-8400-5c309bcea2a8 (at 192.168.203.35@tcp) by administrative request [ 4469.563306] Lustre: Skipped 1 previous similar message [ 4501.491447] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4501.501058] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4501.517150] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4502.499811] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4502.499900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4502.515225] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4504.839122] Lustre: server umount lustre-OST0000 complete [ 4511.200968] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4511.212291] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4511.229777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4511.239876] Lustre: Skipped 1 previous similar message [ 4515.190538] Lustre: server umount lustre-MDT0000 complete [ 4517.861723] LustreError: 153092:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4517.884866] LustreError: 153092:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4519.059381] LustreError: 152341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787382011 with bad export cookie 3749682634121810080 [ 4519.066338] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4519.081712] LustreError: 152341:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4519.548121] Lustre: server umount lustre-MDT0001 complete [ 4531.465585] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4534.191677] Key type lgssc unregistered [ 4534.477916] LNet: 156428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4534.487045] LNetError: 156428:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4534.505439] LNet: Removed LNI 192.168.203.135@tcp [ 4535.366214] Key type .llcrypt unregistered [ 4535.369119] Key type ._llcrypt unregistered [ 4614.132549] Key type ._llcrypt registered [ 4614.138878] Key type .llcrypt registered [ 4614.303659] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4614.975764] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4615.099798] alg: No test for adler32 (adler32-zlib) [ 4616.371338] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4616.641821] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4618.312160] Key type lgssc registered [ 4619.842603] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4628.731898] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4628.757776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4630.225049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4634.718709] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4639.723434] LustreError: 158930:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4639.743979] LustreError: 158930:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4643.380172] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4643.809134] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4648.436692] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4655.644953] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4661.619277] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4668.130168] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4668.434453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4669.503525] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4674.149198] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4674.613042] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4680.851539] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4689.898174] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4689.899154] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4689.907436] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4689.914755] Lustre: Skipped 1 previous similar message [ 4689.938230] LustreError: Skipped 1 previous similar message [ 4695.011068] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4695.022458] Lustre: Skipped 1 previous similar message [ 4700.136972] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4700.141786] Lustre: Skipped 1 previous similar message [ 4704.225802] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4704.415909] Lustre: server umount lustre-OST0000 complete [ 4710.369218] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4710.371173] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4710.383359] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4710.397549] Lustre: Skipped 1 previous similar message [ 4710.407369] Lustre: Skipped 2 previous similar messages [ 4715.492592] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4715.499628] Lustre: Skipped 1 previous similar message [ 4722.144391] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4724.416181] Lustre: server umount lustre-MDT0000 complete [ 4725.734368] LustreError: 159660:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4725.770608] LustreError: 159660:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4728.997262] LustreError: 158911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787382221 with bad export cookie 15448262611698523553 [ 4729.005281] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4729.012031] LustreError: 158911:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4729.584183] Lustre: server umount lustre-MDT0001 complete [ 4741.615359] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4745.176909] Key type lgssc unregistered [ 4745.675113] LNet: 162474:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4745.688163] LNetError: 162474:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4745.704829] LNet: Removed LNI 192.168.203.135@tcp [ 4746.528721] Key type .llcrypt unregistered [ 4746.530723] Key type ._llcrypt unregistered [ 4771.814822] Key type ._llcrypt registered [ 4771.820403] Key type .llcrypt registered [ 4771.924652] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4772.847306] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4772.952177] alg: No test for adler32 (adler32-zlib) [ 4773.920854] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4774.159691] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4775.880195] Key type lgssc registered [ 4776.868349] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4791.873676] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 03:04:42 (1787382282) [ 4806.465626] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 4819.090593] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4857.462325] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4857.495703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4858.735461] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4858.774976] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4858.838848] Lustre: lustre-MDT0000: new disk, initializing [ 4858.897646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4858.918154] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4863.360948] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4872.527107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4872.652847] Lustre: 166735:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4872.713112] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4872.717771] Lustre: Skipped 1 previous similar message [ 4872.813537] Lustre: lustre-MDT0001: new disk, initializing [ 4872.930599] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4872.965274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4872.984270] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4876.935943] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4885.210981] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4889.848542] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4896.255696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4896.518836] Lustre: lustre-OST0000: new disk, initializing [ 4896.522129] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4896.526779] Lustre: 168702:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4896.643123] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4903.516352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4903.524360] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4903.648273] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4904.100962] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 4913.894124] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4924.386716] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4924.391306] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4924.400952] LustreError: Skipped 1 previous similar message [ 4924.410147] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4924.431541] Lustre: Skipped 1 previous similar message [ 4929.649298] Lustre: server umount lustre-OST0000 complete [ 4934.635491] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4934.638640] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4934.643626] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4934.679719] Lustre: Skipped 2 previous similar messages [ 4939.748125] LustreError: 166748:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4939.774453] LustreError: 166748:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 4939.816503] Lustre: server umount lustre-MDT0000 complete [ 4943.240203] LustreError: 166729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787382435 with bad export cookie 15381475457730543304 [ 4943.251177] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4943.255304] LustreError: 166729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4943.514810] Lustre: server umount lustre-MDT0001 complete [ 4953.486888] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 4956.631855] Key type lgssc unregistered [ 4956.922233] LNet: 170611:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4956.936645] LNetError: 170611:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4956.964719] LNet: Removed LNI 192.168.203.135@tcp [ 4957.700978] Key type .llcrypt unregistered [ 4957.707535] Key type ._llcrypt unregistered [ 4983.683830] Key type ._llcrypt registered [ 4983.687025] Key type .llcrypt registered [ 4983.769475] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 4984.769924] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4984.810208] alg: No test for adler32 (adler32-zlib) [ 4985.848687] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 4986.113501] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 4987.944125] Key type lgssc registered [ 4989.132967] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4996.942289] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4996.983120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4998.407070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5002.613694] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 5008.865896] LustreError: 172025:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5008.887100] LustreError: 172025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5010.878199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5011.133955] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5015.836454] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 5022.891308] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5029.402536] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5037.654255] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5038.038743] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5044.234612] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5044.835431] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 5052.623516] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5062.237324] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5062.256450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5062.278920] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5062.284903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.303795] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5063.477277] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5063.519276] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5063.534436] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5063.541895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.560816] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5064.471408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.268194] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5065.273593] Lustre: Skipped 1 previous similar message [ 5065.298704] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5065.312504] Lustre: Skipped 1 previous similar message [ 5065.317990] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5065.322984] Lustre: Skipped 1 previous similar message [ 5065.327759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.336297] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5065.338623] Lustre: Skipped 1 previous similar message [ 5066.389977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.277890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.238594] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5068.248128] Lustre: Skipped 2 previous similar messages [ 5068.268607] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5068.281857] Lustre: Skipped 2 previous similar messages [ 5068.286421] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5068.291467] Lustre: Skipped 2 previous similar messages [ 5068.295929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.314693] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5068.317618] Lustre: Skipped 2 previous similar messages [ 5069.016612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.904794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.809705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.995847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.071921] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5073.079858] Lustre: Skipped 4 previous similar messages [ 5073.094102] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5073.106907] Lustre: Skipped 4 previous similar messages [ 5073.110866] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5073.114127] Lustre: Skipped 4 previous similar messages [ 5073.118797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.130608] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5073.138121] Lustre: Skipped 4 previous similar messages [ 5074.152316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.064061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.105422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.165168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.393538] LustreError: 172045:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5078.407951] LustreError: 172045:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5078.423748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.295322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.238035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.164968] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5081.169971] Lustre: Skipped 7 previous similar messages [ 5081.177948] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5081.187578] Lustre: Skipped 7 previous similar messages [ 5081.191029] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5081.193961] Lustre: Skipped 7 previous similar messages [ 5081.197865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.207142] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5081.210146] Lustre: Skipped 7 previous similar messages [ 5082.150174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.971737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.111542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.916497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.895843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.829147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.760564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.700043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.495222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.549799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.436441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.274197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.993376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.803773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.491285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.260763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.259587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.959474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.756734] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5097.762284] Lustre: Skipped 18 previous similar messages [ 5097.776648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5097.785803] Lustre: Skipped 18 previous similar messages [ 5097.789599] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5097.792981] Lustre: Skipped 18 previous similar messages [ 5097.797682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.807836] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5097.814514] Lustre: Skipped 18 previous similar messages [ 5098.839051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.790550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.974100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.076496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.044936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.135649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.309451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.203514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.203605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.212664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.177766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.117429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.979483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.901711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.745813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.515159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.377359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.341305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.230728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.148411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.998191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.052253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.003440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.140819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.010482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.986425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.042553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.079620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.037720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.938039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.780564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.725643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.555138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.473996] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5130.481224] Lustre: Skipped 33 previous similar messages [ 5130.496820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5130.506283] Lustre: Skipped 33 previous similar messages [ 5130.513959] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5130.519923] Lustre: Skipped 33 previous similar messages [ 5130.528399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.541490] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5130.548271] Lustre: Skipped 33 previous similar messages [ 5131.344563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.159224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.184013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.257569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.138291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.159962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.031459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.906958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.707731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.622392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.427445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.364734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.140514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.108206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.034182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.241534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.425773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.388781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.129201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.137571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.087340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.032170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.922777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.987129] LustreError: 172045:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5152.996449] LustreError: 172045:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5152.998615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.873848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.791107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.742511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.731782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.643341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.703147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.639097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.760490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.663471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.593665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.498719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.515353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.368709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.278487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.114811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.095452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.889585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.671992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.348975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.032697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.916384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.699847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.563919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.486264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.300150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.089230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.843580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.698919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.401163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.147604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.778285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.457554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.087898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.911361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.550320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.293748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.106978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.850439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.495478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.276860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.023351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.910465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.706228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.465877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.547222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.225448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.011273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.853359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.565513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.424562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.185412] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5195.190335] Lustre: Skipped 74 previous similar messages [ 5195.203371] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5195.213361] Lustre: Skipped 74 previous similar messages [ 5195.217711] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5195.223495] Lustre: Skipped 74 previous similar messages [ 5195.228129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.239340] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5195.246155] Lustre: Skipped 74 previous similar messages [ 5196.042314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.743191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.581268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.297455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.058659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.939323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.818299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.716862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.606191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.336313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.070606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.779448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.540234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.152719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.920865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.652557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.473452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.363845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.179097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.917845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.783511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.533270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.498326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.432403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.460326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.437859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.545747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.459569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.518838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.507572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.472550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.306265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.228566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.569693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.447481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.362512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.318866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.248821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.165447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.198963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.351574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.203285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.089619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.034951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.825531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.726142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.545914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.402056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.318659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.367194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.255836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.222935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.254335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.394813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.311906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.132080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.874059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.595800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.283883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.986801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.811652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.721924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.702146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.763996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.752892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.829255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.879450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.911672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.688944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.595728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.443750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.435065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.244959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.269266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.034617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.918482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.752355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.617399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.435065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.238642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.036700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.036626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.943963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.989213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.056571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.964431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.815362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.712757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.510454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.345969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.106925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.881634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.752455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.658910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.465220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.412586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.241803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.129793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.912612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.950745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.914404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.831680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.779941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.841084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.655369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.389514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.230554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.016184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.908102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.844580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.607630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.426416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.407288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.491644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.332476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.366059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.395721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.295706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.228484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.179647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.074579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.835775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.660162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.498432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.329765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.318737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.147748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.955494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.714165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.505087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.379040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.213672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.212213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.076466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.904593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.737763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.495062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.324335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.301941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.341045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.453247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.370924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.480502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.335730] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5323.340832] Lustre: Skipped 143 previous similar messages [ 5323.358881] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5323.378392] Lustre: Skipped 143 previous similar messages [ 5323.386396] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5323.400908] Lustre: Skipped 143 previous similar messages [ 5323.409947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.433966] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5323.441719] Lustre: Skipped 143 previous similar messages [ 5324.366826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.187199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.924106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.624920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.440507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.131957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.957523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.842079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.640173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.484255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.238270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.980910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.741045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.544919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.384965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.137394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.034925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.768702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.793772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.630657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.604994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.396899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.213480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.950615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.876577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.746954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.735629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.648476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.607379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.543155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.499662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.738485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.696502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.681513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.549499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.438635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.333515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.328515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.419074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.380135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.353303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.365785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.269932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.359877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.341365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.326642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.253774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.169406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.089166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.312863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.326835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.340516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.322589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.294571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.152607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.080111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.864400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.767450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.505243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.466134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.493616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.216545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.256892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.053944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.082921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.204051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.510021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.614825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.923695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.844692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.750758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.793580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.989427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.915988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.018384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.934336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.786033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.516628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.229251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.036040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.957282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.844351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.730710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.667456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.566547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.620899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.487078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.233647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.047489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.845593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.660036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.340493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.106222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.943412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.778291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.658711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.604865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.464707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.303050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.378923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.615694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.759292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.764373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.910722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.026726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.957302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.806095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.832304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.810577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.551133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.423521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.302575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.176898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.967396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.884725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.001352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.027156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.101760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.011909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.331733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.449158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.497984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.685563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.553804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.683253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.688355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.760268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.660175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.716386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.563616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.289625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.130794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.134828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.102236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.283583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.187572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.170383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.191690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.390851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.320095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.282188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.077456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.967868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.797706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.658398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.405748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.359838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.314124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.318191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.111893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.897609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.780086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.578132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.671985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.645268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.892261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.988443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.264023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.523292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.421954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.230775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.070527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.034190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.874410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.844627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.641910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.470041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.523933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.598151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.543817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.404396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.320520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.309683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.136459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.898029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.740909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.586808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.320526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.188431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.204455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.021640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.000957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.829974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.787346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.542951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.420445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.193622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.062632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.898845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.592748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.386284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.208601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.992070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.811070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.588643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.501790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.257064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.193911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.007251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.834819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.654722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.537267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.403940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.209372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.066494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.879887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.759846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.587402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.426354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.246078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.017806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.001674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.899868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.928270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.837670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.845865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.040150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.012750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.964945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.830321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.675066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.583293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.540606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.474721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.445461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.216285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.117026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.923974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.708368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.525356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.369763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.177702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.040708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.780264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.867198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.746398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.656294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.740965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.661855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.457318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.281769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.129482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.874246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.682223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.540792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.446917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.172218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.015349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.932420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.812978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.586143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.435223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.184898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.114542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.859719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.804561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.597642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.620725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.530475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.521878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.643393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.746293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.739781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.870407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.969950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.283564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.247078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.328966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.506257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.764038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.832414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.847989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.876744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.841136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.738259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.758679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.874304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.820680] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5579.828698] Lustre: Skipped 277 previous similar messages [ 5579.854977] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5579.869130] Lustre: Skipped 277 previous similar messages [ 5579.874853] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5579.884335] Lustre: Skipped 277 previous similar messages [ 5579.897617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.914231] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5579.920157] Lustre: Skipped 277 previous similar messages [ 5581.017252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.039341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.046628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.021143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.017964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.921799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.884325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.980229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.995699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.030167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.008557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.988351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.874844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.033032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.978035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.948584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.821412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.721138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.631628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.481234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.517407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.466578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.381492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.290277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.256137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.171175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.939325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.666990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.447741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.405598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.294248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.183720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.012984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.830621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.696444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.698301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.689542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.559631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.535132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.305324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.165625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.016083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.971018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.736209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.625734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.388142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.245199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.028640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.837908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.831861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.910854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.702838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.752403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.785499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.750748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.888086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.743946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.613215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.414449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.204492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.966600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.819162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.473035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.421262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.268628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.109553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.953649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.899295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.989511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.996278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.237383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.995702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.893401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.977432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.938454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.290278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.547868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.863957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.884681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.820201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.766561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.711704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.646456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.551760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.537482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.515981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.328217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.319447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.315954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.376356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.449208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.481926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.370211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.352625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.242990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.169809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.041047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.944304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.802636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.119234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.233468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.315115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.362312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.420544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.419817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.379287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.056306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.934614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.711533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.656665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.462800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.276708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.978842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.878514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.588261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.429739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.323223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.233455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.182301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.956508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.796596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.794695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.660605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.472252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.295846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.210622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.049666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.956145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.890554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.745508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.741148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.550385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.480093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.312130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.117797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.963588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.770653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.475186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.498269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.394074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.372372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.276354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.283337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.231469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.328269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.166640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.260300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.199512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.212033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.069608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.975639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.924162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.876181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.853904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.696692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.629843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.489294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.472373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.263379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.204409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.188576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.191758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.154807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.012411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.834959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.639384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.595981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.647728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.593217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.498113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.297457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.360521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.174581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.968152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.745751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.587677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.541194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.520452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.357840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.396647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.426823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.403064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.346359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.303404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.427341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.312495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.231394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.223633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.147703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.069162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.153191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.079325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.081062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.961080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.833937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.738505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.700743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.733901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.680485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.544384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.541254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.455436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.506060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.555586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.302985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.260174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.409520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.349981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.330865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.169599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.151323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.129262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.983627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.930702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.793790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.666563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.545570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.413368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.306794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.221108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.159194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.078241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.923474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.777406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.809168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.917821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.961080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.812259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.641911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.518359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.426589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.418194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.248959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.214420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.069955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.975807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.821821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.666706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.613305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.456650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.318915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.181699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.914629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.768508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.613491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.413987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.143626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.951179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.676395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.607788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.520345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.543361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.485898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.360734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.201866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.168677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.050467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.863938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.659907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.589947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.452732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.328940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.138361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.011482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.797669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.629556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.431808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.188864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.075571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.043820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.016908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.879799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.675903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.730144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.611883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.673420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.646830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.708597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.678130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.712441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.638025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.583746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.398546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.151051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.130276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.046438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.038630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.857827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.090251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.066672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.097932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.084231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.420233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.646855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.660695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.871072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.054567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.973244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.119980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.305673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.099341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.194239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.449475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.811359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.822943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.163760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.305715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.186781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.123731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.192974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.058635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.136659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.119509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.151239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.230836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.271049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.500553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.546808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.602726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.547569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.695318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.456451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.395336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.311858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.246104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.010189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.919367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.867217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.888839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.097584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.216653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.323501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.532404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.771517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.729655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.717356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.684536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.749991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.768731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.863113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.036813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.983349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.029539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.915235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.856799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.753530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.752089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.687242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.828157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.715375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.551100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.294441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.202715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.951803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.687587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.560690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.455250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.293369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.187355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.011420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.134327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.848660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.779531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.835198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.888682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.794692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.714516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.629711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.387787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.280494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.062081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.046978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.878127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.747683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.781850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.734781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.564887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.402688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.390693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.240293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.120200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.045360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.803296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.988484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.937984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.002217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.897410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.876390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.897993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.667421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.483339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.336183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.296447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.119432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.030588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.829632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.636931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.505386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.431426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.408775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.515891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.441112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.277482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.117988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.935341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.754508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.696580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.567521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.383251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.303478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.258469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.212336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.221426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.993760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.863337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.677257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.645401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.442376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.220627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.047317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.885950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.616395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.417278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.313685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.232641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.226626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.167850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.059965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.940877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.884083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.634352] LustreError: 173846:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6420.654906] LustreError: 172045:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7156.209646] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7156.220414] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7156.222824] LustreError: Skipped 1 previous similar message [ 7156.237942] Lustre: Skipped 430 previous similar messages [ 7156.241780] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7157.006026] Lustre: server umount lustre-OST0000 complete [ 7161.825339] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7161.839223] LustreError: Skipped 3 previous similar messages [ 7161.855190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7162.858330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7162.866114] Lustre: Skipped 1 previous similar message [ 7167.450233] Lustre: server umount lustre-MDT0000 complete [ 7167.971598] LustreError: 172025:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7168.017731] LustreError: 172025:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7171.556221] LustreError: 172006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384664 with bad export cookie 11394898564899805715 [ 7171.560599] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7171.573537] LustreError: 172006:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7182.004774] Lustre: server umount lustre-MDT0001 complete [ 7194.586694] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7198.474292] Key type lgssc unregistered [ 7198.819944] LNet: 227288:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7198.832852] LNetError: 227288:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7198.867240] LNet: Removed LNI 192.168.203.135@tcp [ 7200.052304] Key type .llcrypt unregistered [ 7200.055284] Key type ._llcrypt unregistered [ 7227.341832] Key type ._llcrypt registered [ 7227.349327] Key type .llcrypt registered [ 7227.500342] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7228.456180] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7228.474057] alg: No test for adler32 (adler32-zlib) [ 7229.600586] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 7229.966405] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 7231.632235] Key type lgssc registered [ 7232.937775] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7241.398470] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7241.412287] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7242.832421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7247.147864] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7252.973048] LustreError: 228711:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7252.998300] LustreError: 228711:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7256.426049] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7256.780112] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7261.340623] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7269.575884] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7275.971699] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7284.277504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7284.861816] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7287.967423] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7291.879326] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7299.924475] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7308.266695] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7310.118792] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7310.138178] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7310.145177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.169826] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7310.170234] LustreError: 228730:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7312.367406] Lustre: lustre-OST0000: Not available for connect from 192.168.203.35@tcp (stopping) [ 7312.867717] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7312.886409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7312.894370] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7312.920061] Lustre: Skipped 1 previous similar message [ 7317.473884] Lustre: lustre-OST0000: Not available for connect from 192.168.203.35@tcp (stopping) [ 7317.485502] Lustre: Skipped 1 previous similar message [ 7318.548536] Lustre: server umount lustre-OST0000 complete [ 7330.249768] LustreError: 228730:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7352.574222] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7353.012462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7360.470428] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7368.374995] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.400443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7368.403685] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7368.427224] Lustre: Skipped 1 previous similar message [ 7368.595197] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7370.522627] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7376.635068] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fd5085c1000.ost_server_uuid 50 [ 7378.270957] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fd5085c1000.ost_server_uuid in FULL state after 0 sec [ 7384.034925] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7384.043204] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7384.065617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7384.105073] Lustre: Skipped 3 previous similar messages [ 7384.944713] Lustre: server umount lustre-OST0000 complete [ 7389.160180] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7389.178732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7389.188329] Lustre: Skipped 2 previous similar messages [ 7389.207431] Lustre: Skipped 1 previous similar message [ 7395.416871] Lustre: server umount lustre-MDT0000 complete [ 7399.402206] LustreError: 228705:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7399.427762] LustreError: 228705:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7399.824486] LustreError: 228691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787384892 with bad export cookie 10597627773946910471 [ 7399.827713] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7399.846318] LustreError: 228691:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7400.244784] Lustre: server umount lustre-MDT0001 complete [ 7411.309468] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 7413.931986] Key type lgssc unregistered [ 7414.264574] LNet: 233920:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7414.273477] LNetError: 233920:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7414.293041] LNet: Removed LNI 192.168.203.135@tcp [ 7415.041426] Key type .llcrypt unregistered [ 7415.048371] Key type ._llcrypt unregistered [ 7447.773979] Key type ._llcrypt registered [ 7447.777383] Key type .llcrypt registered [ 7447.897403] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7449.196384] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7449.217422] alg: No test for adler32 (adler32-zlib) [ 7450.222922] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 7450.427963] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 7452.152186] Key type lgssc registered [ 7453.261806] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7463.062547] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7463.104890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7464.601079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7468.597834] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7475.178413] LustreError: 235648:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7475.191369] LustreError: 235648:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7476.943150] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7477.343229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7481.750451] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7484.869590] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7491.380380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7491.849481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7497.579428] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7507.461705] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7507.672370] Lustre: lustre-OST0001: new disk, initializing [ 7507.676645] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7507.696137] Lustre: 237957:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7507.840311] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7509.887468] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7514.271327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7514.286070] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7514.439085] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7515.478598] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7526.287590] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7533.400258] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7548.901158] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7548.910374] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7548.926762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7550.434912] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7550.450949] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7550.476343] Lustre: Skipped 2 previous similar messages [ 7555.107642] Lustre: server umount lustre-MDT0000 complete [ 7555.556381] LustreError: 236378:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7555.591167] LustreError: 236378:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7559.171440] LustreError: 238843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385051 with bad export cookie 6809494529765083606 [ 7559.205119] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7559.723608] Lustre: server umount lustre-MDT0001 complete [ 7574.277810] Lustre: server umount lustre-OST0000 complete [ 7576.557296] Lustre: 234913:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787385053/real 1787385053] req@ffff9995bc533800 x1874208954084224/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787385069 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7576.607439] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7578.996207] Lustre: server umount lustre-OST0001 complete [ 7604.651490] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7615.504365] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7615.971463] LustreError: 241064:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7616.109811] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7620.643597] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7621.089475] LustreError: 241064:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7621.115560] LustreError: 241064:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7626.228886] LustreError: 241065:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7629.700532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7630.069162] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7635.006489] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7638.025890] Lustre: 242207:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7644.436169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7652.194718] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7655.150954] LustreError: 242563:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7661.248659] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7661.533308] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7661.538497] Lustre: Skipped 1 previous similar message [ 7662.586085] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7662.619099] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7667.744063] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7675.458596] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7679.165455] Lustre: 244078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7691.756382] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7691.766704] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7691.791416] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7691.796898] Lustre: Skipped 3 previous similar messages [ 7696.486761] Lustre: server umount mylustre-MDT0000 complete [ 7698.420725] LustreError: 241797:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7698.440788] LustreError: 241797:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7700.307706] LustreError: 241041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385192 with bad export cookie 6809494529765086413 [ 7700.313703] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7700.322497] LustreError: 241041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7700.798803] Lustre: server umount mylustre-MDT0001 complete [ 7715.333905] Lustre: server umount mylustre-OST0000 complete [ 7728.713810] Lustre: server umount mylustre-OST0001 complete [ 7755.441853] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7767.350536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7767.945331] LustreError: 246387:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7768.050200] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7773.526502] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7783.168040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7789.734572] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7793.038684] Lustre: 247531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7801.363298] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7809.971506] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7813.362238] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7823.623362] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7829.509603] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7833.198994] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7841.698121] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7845.643204] Lustre: 249405:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7855.076672] Lustre: tfs-MDT0000-lwp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7855.078754] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7855.096993] Lustre: Skipped 6 previous similar messages [ 7855.121720] Lustre: Skipped 7 previous similar messages [ 7855.863858] Lustre: server umount tfs-MDT0000 complete [ 7859.878699] LustreError: 246364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385352 with bad export cookie 6809494529765088303 [ 7859.885210] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7859.889150] LustreError: 246364:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7860.194879] LustreError: 246382:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7860.213126] LustreError: 246382:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 13 previous similar messages [ 7860.370852] Lustre: server umount tfs-MDT0001 complete [ 7866.944784] Lustre: server umount tfs-OST0000 complete [ 7871.219474] Lustre: server umount tfs-OST0001 complete [ 7894.815412] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 7906.460750] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7907.153494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7907.162306] Lustre: Skipped 3 previous similar messages [ 7912.572672] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7922.596332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7927.623181] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7930.878608] Lustre: 252793:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7938.722398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7946.032350] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7951.426065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7956.658149] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7962.105546] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7964.571819] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 7973.013180] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7977.145676] Lustre: 254660:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7984.679343] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 03:57:55 (1787385475) [ 8003.062873] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8003.075130] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8003.085239] Lustre: Skipped 4 previous similar messages [ 8003.111970] Lustre: Skipped 3 previous similar messages [ 8007.791219] Lustre: server umount lustre-MDT0000 complete [ 8008.163122] LustreError: 251648:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8008.175148] LustreError: 251648:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 8011.531790] LustreError: 251629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385504 with bad export cookie 6809494529765089969 [ 8011.542172] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8011.552968] LustreError: 251629:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8013.290407] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8013.301449] Lustre: Skipped 1 previous similar message [ 8018.068888] Lustre: server umount lustre-MDT0001 complete [ 8037.344133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8037.420327] Lustre: server umount lustre-OST0000 complete [ 8047.519716] Lustre: server umount lustre-OST0001 complete [ 8055.298477] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8067.005512] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8108.971229] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8108.971229] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8108.971229] [ 8109.000227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8109.400864] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8109.436812] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8109.572619] Lustre: lustre-MDT0000: new disk, initializing [ 8109.697924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8109.711354] Lustre: Skipped 3 previous similar messages [ 8109.751121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8115.866831] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8127.456348] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8127.456348] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8127.456348] [ 8127.541264] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8127.667685] Lustre: 257853:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8127.735491] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8127.744249] Lustre: Skipped 1 previous similar message [ 8127.837736] Lustre: lustre-MDT0001: new disk, initializing [ 8127.961293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8127.979988] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8133.036686] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8146.270109] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8146.770156] Lustre: lustre-OST0000: new disk, initializing [ 8146.774981] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8146.781211] Lustre: 259435:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8148.257494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8148.285877] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8148.490777] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8154.734162] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8166.242030] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8166.483346] Lustre: lustre-OST0001: new disk, initializing [ 8166.494683] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8166.510722] Lustre: 260306:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8167.835577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8167.852145] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8167.967311] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8173.633615] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8185.587218] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 04:01:16 (1787385676) [ 8201.707443] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 04:01:32 (1787385692) [ 8219.124424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8219.139911] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8219.150608] Lustre: Skipped 4 previous similar messages [ 8219.164392] Lustre: Skipped 3 previous similar messages [ 8222.796075] Lustre: server umount lustre-OST0000 complete [ 8233.718104] Lustre: server umount lustre-MDT0000 complete [ 8238.099382] LustreError: 257845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385730 with bad export cookie 6809494529765091516 [ 8238.100498] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8238.108891] LustreError: 257845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8244.682350] Lustre: server umount lustre-MDT0001 complete [ 8256.748343] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8268.662380] Lustre: server umount lustre-OST0001 complete [ 8275.339955] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8290.594556] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8328.434509] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8328.697781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8328.734868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8328.844492] Lustre: lustre-MDT0000: new disk, initializing [ 8328.966911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8334.381992] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8345.296934] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8345.417702] Lustre: 264872:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8345.527912] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8345.533285] Lustre: Skipped 1 previous similar message [ 8345.605978] Lustre: lustre-MDT0001: new disk, initializing [ 8345.786903] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8345.804829] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8350.216656] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8359.874664] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8365.400514] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8373.305654] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8373.655323] Lustre: 266841:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8373.738365] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8373.742379] Lustre: Skipped 5 previous similar messages [ 8375.495751] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8375.589945] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8379.549534] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8388.551599] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8396.259087] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8396.265573] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8396.289589] Lustre: Skipped 4 previous similar messages [ 8396.308391] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8396.323677] Lustre: Skipped 6 previous similar messages [ 8402.328326] Lustre: server umount lustre-OST0000 complete [ 8420.320457] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8420.680252] Lustre: server umount lustre-MDT0000 complete [ 8421.859801] LustreError: 265755:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8421.894583] LustreError: 265755:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 14 previous similar messages [ 8425.048136] LustreError: 264863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787385917 with bad export cookie 6809494529765092993 [ 8425.077162] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8425.092375] LustreError: 264863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8425.514813] Lustre: server umount lustre-MDT0001 complete [ 8437.863155] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8440.985175] Key type lgssc unregistered [ 8441.227918] LNet: 268603:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8441.234301] LNetError: 268603:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8441.259282] LNet: Removed LNI 192.168.203.135@tcp [ 8442.123158] Key type .llcrypt unregistered [ 8442.129971] Key type ._llcrypt unregistered [ 8467.713507] Key type ._llcrypt registered [ 8467.716048] Key type .llcrypt registered [ 8467.860354] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8469.145891] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8469.184768] alg: No test for adler32 (adler32-zlib) [ 8470.219803] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 8470.444339] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 8472.096181] Key type lgssc registered [ 8473.513560] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8481.814460] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8481.837451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8483.341807] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8488.382603] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8492.519870] LustreError: 270081:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8492.537358] LustreError: 270081:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8497.634198] LustreError: 270082:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8497.773144] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8498.261922] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8502.713848] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8510.773686] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8516.943433] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8524.633433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8525.001795] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8528.131119] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8531.897332] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8539.699329] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8540.727172] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8543.714384] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8543.728383] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8543.748307] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8547.815435] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8547.816403] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8547.817018] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8549.496597] Lustre: server umount lustre-OST0000 complete [ 8554.469950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8554.485269] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8554.528311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8554.531123] Lustre: Skipped 1 previous similar message [ 8558.050924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8558.053311] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8560.903802] Lustre: server umount lustre-MDT0000 complete [ 8563.169647] LustreError: 270082:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8563.206637] LustreError: 270082:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8565.678146] LustreError: 270061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787386058 with bad export cookie 9035541991884746978 [ 8565.693739] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8565.699204] LustreError: 270061:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8566.295232] Lustre: server umount lustre-MDT0001 complete [ 8578.937245] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8582.662996] Key type lgssc unregistered [ 8583.056649] LNet: 273666:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8583.066349] LNetError: 273666:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8583.075669] LNet: Removed LNI 192.168.203.135@tcp [ 8584.276930] Key type .llcrypt unregistered [ 8584.285068] Key type ._llcrypt unregistered [ 8603.365420] Key type ._llcrypt registered [ 8603.372354] Key type .llcrypt registered [ 8603.518282] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8604.794530] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8604.830143] alg: No test for adler32 (adler32-zlib) [ 8606.338952] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 8606.690629] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 8608.464340] Key type lgssc registered [ 8609.942321] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8621.234124] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8621.265748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8636.832525] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8641.952344] LustreError: 274860:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.135@tcp: failed processing log, type 4: rc = -110 [ 8668.643079] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8676.568248] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8684.335882] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8687.153610] Lustre: server umount lustre-OST0000 complete [ 8704.829466] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 8708.185523] Key type lgssc unregistered [ 8708.625756] LNet: 276492:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8708.638265] LNetError: 276492:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8708.672261] LNet: Removed LNI 192.168.203.135@tcp [ 8709.702269] Key type .llcrypt unregistered [ 8709.707794] Key type ._llcrypt unregistered [ 8749.340261] Key type ._llcrypt registered [ 8749.342531] Key type .llcrypt registered [ 8749.483026] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8750.552396] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8750.657690] alg: No test for adler32 (adler32-zlib) [ 8751.708955] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 8751.917405] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 8753.593579] Key type lgssc registered [ 8754.671512] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8767.033282] loop0: detected capacity change from 0 to 204800000 [ 8767.136991] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8767.148927] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8767.313980] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8768.535682] loop0: detected capacity change from 0 to 204800000 [ 8768.762287] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8768.783923] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8769.238649] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8770.278856] loop0: detected capacity change from 0 to 204800000 [ 8770.447314] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8770.468150] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8770.651302] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8771.684472] loop0: detected capacity change from 0 to 204800000 [ 8771.878855] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8772.977429] loop0: detected capacity change from 0 to 204800000 [ 8773.021490] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8778.153536] loop0: detected capacity change from 0 to 204800000 [ 8778.233132] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8782.908307] loop0: detected capacity change from 0 to 204800000 [ 8782.956577] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8788.345538] loop0: detected capacity change from 0 to 204800000 [ 8788.417872] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8793.014050] loop0: detected capacity change from 0 to 204800000 [ 8793.130775] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8793.178351] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8793.290745] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8793.313087] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8793.330222] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8793.352845] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8793.370970] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8793.384975] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8793.418438] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8793.440673] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8793.455583] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8793.463459] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8793.470879] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8793.485513] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8793.499339] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8793.520258] Lustre: 279921:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8799.872158] Lustre: server umount MGS complete [ 8800.882183] loop0: detected capacity change from 0 to 204800000 [ 8800.980865] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8801.510119] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8801.566461] Lustre: lustre-MDT0000: Aborting client recovery [ 8802.486291] loop1: detected capacity change from 0 to 204800000 [ 8802.603103] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8802.724319] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8802.739724] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8802.759624] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8802.771892] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8802.782957] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8802.793375] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8802.801883] Lustre: 280353:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8802.823634] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8803.012391] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8803.076565] Lustre: lustre-MDT0001: Aborting client recovery [ 8804.076504] loop2: detected capacity change from 0 to 204800000 [ 8804.300479] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8804.456231] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8804.465714] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8804.475409] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8804.482643] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8804.492288] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8804.507293] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8804.515831] Lustre: 280432:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8804.645709] Lustre: Found index 0 for lustre-OST0000, updating log [ 8804.693213] Lustre: 280451:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8804.746283] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8804.769206] Lustre: lustre-OST0000: Aborting recovery [ 8805.688481] loop3: detected capacity change from 0 to 204800000 [ 8805.913353] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8806.013892] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8806.033959] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8806.048017] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8806.075247] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8806.093243] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8806.105600] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8806.119553] Lustre: 280516:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8806.140458] Lustre: Found index 1 for lustre-OST0001, updating log [ 8806.190835] Lustre: 280521:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8806.308902] Lustre: lustre-OST0001: Aborting recovery [ 8811.446440] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8811.451660] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8811.465446] LustreError: 280297:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8811.508131] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8811.523980] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8812.465111] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8812.471538] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8815.442252] LustreError: 280263:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 883us: rc = -2 [ 8815.944619] LustreError: 280262:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 976us: rc = -2 [ 8815.962048] LustreError: 280262:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 20 previous similar messages [ 8828.899946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8828.912354] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8828.920033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8833.006309] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8833.010034] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8833.036968] Lustre: Skipped 1 previous similar message [ 8838.133667] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8838.142461] Lustre: Skipped 6 previous similar messages [ 8843.232621] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8843.244215] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8843.252487] Lustre: Skipped 3 previous similar messages [ 8843.484495] Lustre: server umount lustre-MDT0000 complete [ 8844.251825] LustreError: 280247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787386336 with bad export cookie 7911785034561857790 [ 8844.257503] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8844.263298] LustreError: 280247:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8848.355408] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8848.361021] LustreError: 280753:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8848.361208] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8848.378397] Lustre: Skipped 2 previous similar messages [ 8848.400914] LustreError: 280753:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8849.381573] LustreError: 280261:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8849.405801] LustreError: 280261:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8850.720917] Lustre: server umount lustre-MDT0001 complete [ 8857.804647] Lustre: server umount lustre-OST0000 complete [ 8865.120581] Lustre: server umount lustre-OST0001 complete [ 8874.112871] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 04:12:45 (1787386365) [ 8903.978350] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 8915.850566] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 8957.327312] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8957.495791] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8957.518097] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8957.579966] Lustre: lustre-MDT0000: new disk, initializing [ 8957.639948] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8957.648178] Lustre: Skipped 1 previous similar message [ 8957.677761] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8962.422498] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8973.751077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8973.891483] Lustre: 284594:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8973.984484] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8973.990963] Lustre: Skipped 1 previous similar message [ 8974.150370] Lustre: lustre-MDT0001: new disk, initializing [ 8974.320584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8974.397597] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8974.403644] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8979.152795] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 8988.897727] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8994.902897] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9001.934498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9002.340502] Lustre: lustre-OST0000: new disk, initializing [ 9002.348986] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9002.358035] Lustre: 286562:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9002.471300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9009.247372] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9009.254723] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9009.353244] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9009.391523] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9018.042598] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9022.935434] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9036.503864] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9043.901783] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9048.361432] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9096.679106] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9096.681941] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9096.701688] LustreError: Skipped 1 previous similar message [ 9096.702987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9096.702995] Lustre: Skipped 3 previous similar messages [ 9096.754524] Lustre: Skipped 1 previous similar message [ 9098.566547] Lustre: server umount lustre-OST0000 complete [ 9106.913770] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9106.926549] Lustre: Skipped 1 previous similar message [ 9109.119753] Lustre: server umount lustre-MDT0000 complete [ 9112.033957] LustreError: 284604:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9112.070890] LustreError: 284604:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9113.289156] LustreError: 284584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787386605 with bad export cookie 7911785034561860506 [ 9113.299380] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9113.301949] LustreError: 284584:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9113.656909] Lustre: server umount lustre-MDT0001 complete [ 9123.234470] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9127.889975] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9133.268832] Lustre: server umount MGS complete [ 9145.011390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9145.601722] LustreError: 290526:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9145.629606] LustreError: 290539:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9145.702454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9150.950808] LustreError: 290527:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9151.065444] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9161.190030] LustreError: 290526:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9161.225337] LustreError: 290526:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9162.189867] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9167.150719] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9175.451897] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9181.875469] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9189.008822] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9189.359517] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9189.369040] Lustre: Skipped 1 previous similar message [ 9196.506695] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9204.905689] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9233.393244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9233.397797] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9233.411983] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9233.411990] Lustre: Skipped 3 previous similar messages [ 9238.829605] Lustre: server umount lustre-OST0000 complete [ 9248.861511] Lustre: server umount lustre-MDT0000 complete [ 9252.850876] LustreError: 293362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787386745 with bad export cookie 7911785034561862347 [ 9252.859431] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9252.866629] LustreError: 293362:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9253.174418] Lustre: server umount lustre-MDT0001 complete [ 9264.782390] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 9267.949930] Key type lgssc unregistered [ 9268.318659] LNet: 294471:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9268.344843] LNetError: 294471:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9268.369285] LNet: Removed LNI 192.168.203.135@tcp [ 9269.251055] Key type .llcrypt unregistered [ 9269.253598] Key type ._llcrypt unregistered [ 9316.978487] Key type ._llcrypt registered [ 9316.980686] Key type .llcrypt registered [ 9317.222642] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 9334.280697] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9335.629246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9335.655365] alg: No test for adler32 (adler32-zlib) [ 9337.126227] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 9337.336694] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 9338.968198] Key type lgssc registered [ 9340.531667] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9385.959475] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9386.014459] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9387.563436] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9387.628749] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9387.808106] Lustre: lustre-MDT0000: new disk, initializing [ 9388.030936] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9388.088396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9397.226345] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9410.000062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9410.165836] Lustre: 298431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9410.220491] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9410.225243] Lustre: Skipped 1 previous similar message [ 9410.312381] Lustre: lustre-MDT0001: new disk, initializing [ 9410.440314] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9410.527975] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9410.541491] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9416.034459] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9428.239864] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9435.904138] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9443.863287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9444.156584] Lustre: lustre-OST0000: new disk, initializing [ 9444.168709] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9444.177733] Lustre: 300405:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9444.319054] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9451.584956] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9451.603959] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9451.744406] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9452.218515] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9462.260502] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9469.496738] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9482.678316] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9489.584352] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9533.922345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9533.942597] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9533.947429] Lustre: Skipped 1 previous similar message [ 9533.960771] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9533.978108] LustreError: Skipped 1 previous similar message [ 9538.786342] Lustre: server umount lustre-OST0000 complete [ 9543.655837] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9543.670488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9543.694560] Lustre: Skipped 1 previous similar message [ 9543.700416] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9549.044180] Lustre: server umount lustre-MDT0000 complete [ 9552.914523] LustreError: 301261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787387045 with bad export cookie 321908057706191731 [ 9552.926241] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9552.926717] LustreError: 301261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9553.379459] Lustre: server umount lustre-MDT0001 complete [ 9562.401054] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9567.356852] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9572.388250] Lustre: server umount MGS complete [ 9584.225820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9584.736097] LustreError: 304317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9584.752697] LustreError: 304317:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9584.775849] LustreError: 304330:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9584.850833] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9590.056698] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9590.247696] LustreError: 304318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9594.350262] LustreError: 304317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9599.467122] LustreError: 304318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9599.950053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9600.377727] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9605.356608] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9614.101124] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9621.213698] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9628.467466] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9628.896315] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9635.476505] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9644.681089] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9681.380458] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9681.383763] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9681.387892] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9681.387900] Lustre: Skipped 2 previous similar messages [ 9681.406708] Lustre: Skipped 2 previous similar messages [ 9681.451586] LustreError: Skipped 1 previous similar message [ 9686.515520] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9686.522979] Lustre: Skipped 2 previous similar messages [ 9691.629711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9691.641486] Lustre: Skipped 1 previous similar message [ 9692.128185] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9692.431933] Lustre: server umount lustre-OST0000 complete [ 9697.762093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9697.776432] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9701.862573] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9701.863713] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9701.886595] Lustre: Skipped 1 previous similar message [ 9703.202669] Lustre: server umount lustre-MDT0000 complete [ 9706.990701] LustreError: 304313:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9707.018167] LustreError: 304313:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9707.814359] LustreError: 304297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787387200 with bad export cookie 321908057706193565 [ 9707.815632] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9707.823207] LustreError: 304297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9708.327211] Lustre: server umount lustre-MDT0001 complete [ 9721.465934] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [ 9724.791475] Key type lgssc unregistered [ 9725.176610] LNet: 308264:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9725.182114] LNetError: 308264:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9725.225735] LNet: Removed LNI 192.168.203.135@tcp [ 9726.438712] Key type .llcrypt unregistered [ 9726.440225] Key type ._llcrypt unregistered [ 9756.863058] Key type ._llcrypt registered [ 9756.866506] Key type .llcrypt registered [ 9756.957478] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [ 9772.111045] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9773.316092] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9773.396121] alg: No test for adler32 (adler32-zlib) [ 9774.477761] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [ 9774.857836] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [ 9776.584220] Key type lgssc registered [ 9777.991479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9809.253661] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [ 9832.802548] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9832.822141] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9834.124154] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9834.221562] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9834.344276] Lustre: lustre-MDT0000: new disk, initializing [ 9834.581075] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9834.622976] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9840.690567] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [ 9856.451805] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9856.876916] Lustre: lustre-OST0000: new disk, initializing [ 9856.883643] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9856.888884] Lustre: Skipped 1 previous similar message [ 9856.896476] Lustre: 312323:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9856.997276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9863.844540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9863.861485] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9863.912682] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9864.847541] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10862.065319] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10862.085488] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10865.454150] Lustre: server umount lustre-MDT0000 complete [10869.582207] LustreError: 311507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388362 with bad export cookie 9085991965768806387 [10869.596984] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10869.599235] LustreError: 311507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10869.985686] Lustre: server umount lustre-OST0000 complete [10901.564554] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [10913.279845] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [10952.355922] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10952.673130] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10952.720497] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10952.856187] Lustre: lustre-MDT0000: new disk, initializing [10952.944660] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10952.965721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10957.846537] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10967.290203] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10967.422479] Lustre: 316700:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10967.462948] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10967.469196] Lustre: Skipped 1 previous similar message [10967.533055] Lustre: lustre-MDT0001: new disk, initializing [10967.615667] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10967.657893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10967.672069] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10971.659679] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [10981.692269] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10989.460879] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10997.517390] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10997.795388] Lustre: lustre-OST0000: new disk, initializing [10997.799830] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10997.807707] Lustre: 318668:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10997.931370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11005.108329] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11005.497951] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11005.505708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11005.619778] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11015.088808] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11019.587172] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11019.811445] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11024.817185] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11025.049354] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11031.523234] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11031.527080] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11031.529283] LustreError: Skipped 1 previous similar message [11031.538172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11031.549192] Lustre: Skipped 1 previous similar message [11033.475491] Lustre: server umount lustre-OST0000 complete [11039.203688] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11039.221321] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11039.238690] Lustre: Skipped 1 previous similar message [11039.245623] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11041.764260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11041.765397] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11043.575509] Lustre: server umount lustre-MDT0000 complete [11047.484923] LustreError: 316692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388540 with bad export cookie 9085991965771747213 [11047.494155] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11047.500407] LustreError: 316692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11048.028852] Lustre: server umount lustre-MDT0001 complete [11052.456943] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11054.609879] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 04:49:05 (1787388545) [11063.874780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11064.468173] LustreError: 320981:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11064.502310] LustreError: 320981:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11064.644345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11069.921230] LustreError: 320982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11069.932142] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11075.046870] LustreError: 320981:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11080.161602] LustreError: 320982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11081.030541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11081.483433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11085.920644] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11093.546714] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11099.351972] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11105.901852] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11106.188324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11112.349417] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11119.571310] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11125.914545] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11126.080406] Lustre: lustre-OST0001: new disk, initializing [11126.083628] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11126.089516] Lustre: 323817:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11126.165961] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11131.723378] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11131.968571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11131.980558] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11132.043694] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11132.052330] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11133.093753] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11134.180132] LustreError: 324189:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11134.196536] LustreError: 324189:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11136.228237] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11136.236088] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11140.330558] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11140.348478] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [11141.546625] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11141.870304] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11147.976547] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9fd505ec4000.ost_server_uuid 50 [11148.582995] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11148.604997] LustreError: 324194:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [11150.092141] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9fd505ec4000.ost_server_uuid in FULL state after 0 sec [11156.925410] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9fd505ec4000.ost_server_uuid 50 [11158.960597] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9fd505ec4000.ost_server_uuid in FULL state after 0 sec [11161.095097] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11176.417092] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11176.418842] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11176.426651] LustreError: Skipped 1 previous similar message [11176.440139] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11176.448796] Lustre: Skipped 2 previous similar messages [11179.672446] Lustre: server umount lustre-OST0001 complete [11183.594699] LustreError: 322801:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11183.605834] LustreError: 322801:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11187.169519] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11187.184680] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11187.206028] Lustre: Skipped 1 previous similar message [11187.214981] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11187.218968] Lustre: Skipped 3 previous similar messages [11190.869880] Lustre: server umount lustre-OST0000 complete [11194.345458] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11200.376568] Lustre: server umount lustre-MDT0000 complete [11204.066624] LustreError: 320977:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11204.082329] LustreError: 320977:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11204.314338] LustreError: 320964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388696 with bad export cookie 9085991965771748095 [11204.325978] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11204.327705] LustreError: 320964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11204.663598] Lustre: server umount lustre-MDT0001 complete [11215.120916] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [11217.406888] Key type lgssc unregistered [11217.633027] LNet: 326506:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11217.639118] LNetError: 326506:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11217.663575] LNet: Removed LNI 192.168.203.135@tcp [11218.420232] Key type .llcrypt unregistered [11218.424589] Key type ._llcrypt unregistered [11243.059956] Key type ._llcrypt registered [11243.062968] Key type .llcrypt registered [11243.147461] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [11243.965331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11243.988288] alg: No test for adler32 (adler32-zlib) [11245.062892] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [11245.287994] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [11246.961673] Key type lgssc registered [11248.125554] Lustre: Echo OBD driver; http://www.lustre.org/ [11255.620606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11255.676055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11257.250590] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11262.380081] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11266.540455] LustreError: 327931:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11266.557989] LustreError: 327931:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11269.340977] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11271.663675] LustreError: 327932:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11275.368059] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11275.691340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11279.786307] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11288.220732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11288.552092] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11294.465923] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11300.848655] LustreError: 329610:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11301.665741] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11301.891707] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11306.980505] LustreError: 329612:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11307.007165] LustreError: 329612:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11307.647166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11307.853694] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11313.181865] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11314.244064] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11320.948589] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [11328.206695] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11328.405803] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11436.001524] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11436.002193] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11436.004744] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11436.012234] LustreError: Skipped 1 previous similar message [11436.044075] Lustre: Skipped 1 previous similar message [11440.825741] Lustre: server umount lustre-OST0001 complete [11441.129183] LustreError: 329610:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11441.145914] LustreError: 329610:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11449.313665] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11449.323719] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11449.343613] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11449.356825] Lustre: Skipped 1 previous similar message [11450.337576] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11450.346742] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11450.360230] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11451.366311] LustreError: 329610:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11451.391185] LustreError: 329610:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11456.482964] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11456.491898] Lustre: Skipped 3 previous similar messages [11461.088204] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11461.252966] Lustre: server umount lustre-OST0000 complete [11466.724790] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11466.745312] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11466.752284] Lustre: Skipped 1 previous similar message [11466.766899] Lustre: Skipped 1 previous similar message [11471.751981] Lustre: server umount lustre-MDT0000 complete [11471.852757] LustreError: 327926:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11471.900379] LustreError: 327926:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11476.042317] LustreError: 331618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787388968 with bad export cookie 14749624542176857673 [11476.047801] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11476.048252] LustreError: 331618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11476.496409] Lustre: server umount lustre-MDT0001 complete [11487.668588] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [11490.623959] Key type lgssc unregistered [11491.025033] LNet: 332874:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11491.040308] LNetError: 332874:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11491.071245] LNet: Removed LNI 192.168.203.135@tcp [11492.141957] Key type .llcrypt unregistered [11492.145795] Key type ._llcrypt unregistered [11538.855528] Key type ._llcrypt registered [11538.861928] Key type .llcrypt registered [11538.979406] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [11539.972539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11540.139078] alg: No test for adler32 (adler32-zlib) [11541.205822] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [11541.554829] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [11543.224216] Key type lgssc registered [11544.194460] Lustre: Echo OBD driver; http://www.lustre.org/ [11552.395706] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11552.414644] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11553.770513] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11557.750861] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11564.007326] LustreError: 335165:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11564.029270] LustreError: 335165:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11565.448636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11565.902744] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11569.917985] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11572.490451] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11578.913348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11579.214848] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11585.779588] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11590.462092] LustreError: 336657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11594.285688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11594.528432] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11598.658080] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11598.664979] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11600.597388] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11608.138513] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11613.436641] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11627.491170] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11627.500508] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11627.511510] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11629.545945] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11629.554495] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11629.558474] Lustre: Skipped 1 previous similar message [11629.573859] Lustre: Skipped 1 previous similar message [11631.663109] Lustre: server umount lustre-MDT0000 complete [11634.664852] LustreError: 335165:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11634.680953] LustreError: 335165:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11635.883429] LustreError: 335147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389128 with bad export cookie 13098942585799778331 [11635.885418] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11635.902516] LustreError: 335147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11636.263940] Lustre: server umount lustre-MDT0001 complete [11650.556387] Lustre: server umount lustre-OST0000 complete [11665.300602] Lustre: server umount lustre-OST0001 complete [11683.035750] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [11693.002843] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11693.509677] LustreError: 340432:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11693.580304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11698.448876] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11698.660817] LustreError: 340433:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11706.485698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11706.778219] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11711.463880] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11714.448394] Lustre: 341572:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11720.355866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11726.508873] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11732.025058] LustreError: 341925:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11732.054727] LustreError: 341925:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11734.956747] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11735.248622] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11735.255762] Lustre: Skipped 1 previous similar message [11738.378409] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11738.381983] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11740.702308] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11747.371869] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11750.360561] Lustre: 343444:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11753.955458] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11753.969827] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11753.978055] Lustre: Skipped 4 previous similar messages [11753.991506] Lustre: Skipped 3 previous similar messages [11759.075559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11759.083506] Lustre: Skipped 2 previous similar messages [11759.515348] Lustre: server umount lustre-MDT0000 complete [11762.516305] LustreError: 340412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389255 with bad export cookie 13098942585799780032 [11762.526186] LustreError: 340412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11762.529586] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11762.837255] Lustre: server umount lustre-MDT0001 complete [11776.340779] Lustre: server umount lustre-OST0000 complete [11789.491727] Lustre: server umount lustre-OST0001 complete [11793.562600] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11795.219277] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11796.974136] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:01:27 (1787389287) [11815.525201] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [11823.170690] loop0: detected capacity change from 0 to 1099511627776 [11823.850190] SGI XFS with ACLs, security attributes, quota, no debug enabled [11823.854915] *** VALIDATE xfs *** [11823.864451] XFS (loop0): Mounting V5 Filesystem [11823.888460] XFS (loop0): Ending clean mount [11825.035380] loop1: detected capacity change from 0 to 18691697672192 [11915.896907] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [11925.899491] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [11960.683270] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11960.882423] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11960.901215] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11960.957509] Lustre: lustre-MDT0000: new disk, initializing [11961.012937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11961.030543] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11964.967614] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11974.387781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11974.527576] Lustre: 348589:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11974.625924] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11974.631762] Lustre: Skipped 1 previous similar message [11974.714781] Lustre: lustre-MDT0001: new disk, initializing [11974.842634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11974.856058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11979.030746] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [11987.302903] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11993.300795] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12000.317708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12000.613281] Lustre: lustre-OST0000: new disk, initializing [12000.618347] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12000.623849] Lustre: 350557:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12002.676954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12002.696214] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12002.831946] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12006.366640] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12015.922908] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12021.514227] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12021.756390] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12026.856394] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12027.047788] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12033.505651] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12033.510154] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12033.521515] LustreError: Skipped 1 previous similar message [12033.523230] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12033.584925] Lustre: Skipped 1 previous similar message [12036.165459] Lustre: server umount lustre-OST0000 complete [12041.699912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12041.710111] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12041.735315] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12041.738493] Lustre: Skipped 1 previous similar message [12047.714556] Lustre: server umount lustre-MDT0000 complete [12048.866023] LustreError: 348595:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12048.877128] LustreError: 348595:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12051.664275] LustreError: 348581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389544 with bad export cookie 13098942585799781579 [12051.671295] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12051.680393] LustreError: 348581:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12052.212813] Lustre: server umount lustre-MDT0001 complete [12062.053485] XFS (loop0): Unmounting Filesystem [12066.583949] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 05:05:57 (1787389557) [12073.637696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12074.307234] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12074.312213] Lustre: Skipped 2 previous similar messages [12079.207833] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12083.685242] LustreError: 353117:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12083.694672] LustreError: 353117:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12089.660989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12094.584435] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12102.660333] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12108.946693] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12114.981748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12122.084517] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12129.466245] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12144.103815] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12144.112557] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12144.125519] Lustre: Skipped 1 previous similar message [12144.132135] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12144.136781] Lustre: Skipped 2 previous similar messages [12147.754422] Lustre: server umount lustre-OST0000 complete [12151.777294] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12151.795087] LustreError: Skipped 1 previous similar message [12157.814651] Lustre: server umount lustre-MDT0000 complete [12160.482475] LustreError: 353117:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12160.497824] LustreError: 353117:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12161.641326] LustreError: 353098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787389654 with bad export cookie 13098942585799782461 [12161.645156] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12161.655788] LustreError: 353098:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12162.049881] Lustre: server umount lustre-MDT0001 complete [12172.958774] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [12175.368985] Key type lgssc unregistered [12175.621240] LNet: 356747:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12175.633959] LNetError: 356747:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12175.651729] LNet: Removed LNI 192.168.203.135@tcp [12176.521152] Key type .llcrypt unregistered [12176.522993] Key type ._llcrypt unregistered [12203.338843] Key type ._llcrypt registered [12203.343924] Key type .llcrypt registered [12203.533547] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [12204.573241] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12204.737961] alg: No test for adler32 (adler32-zlib) [12205.946678] Lustre: Lustre: Build Version: 2.17.54_117_g8b0669c [12206.308686] LNet: Added LNI 192.168.203.135@tcp [8/256/0/180] [12208.057277] Key type lgssc registered [12209.614606] Lustre: Echo OBD driver; http://www.lustre.org/ [12217.794082] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12217.839174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12219.286912] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12223.723649] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12228.587843] LustreError: 358317:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12228.617495] LustreError: 358317:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12233.134029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12233.616171] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12238.150678] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12244.892328] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12249.617670] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12254.831627] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12255.079074] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12260.210615] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12266.748315] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12274.134591] Lustre: server umount lustre-MDT0001 complete [12274.658624] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12274.679692] LustreError: 359045:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12274.704342] LustreError: 359045:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12279.751174] LustreError: 358312:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12284.713413] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12284.801442] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12284.806320] Lustre: Found index 1 for lustre-MDT0001, updating log [12284.859719] Lustre: 358305:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12284.891840] LustreError: 359751:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12284.920260] LustreError: 359751:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12285.180741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12289.564662] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12290.546686] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12295.166227] LustreError: 361916:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12295.190270] LustreError: 361916:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12295.207286] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12295.267338] LustreError: 361926:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -17 [12295.277555] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12295.277555] [12597.220666] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.35@tcp (stopping) [12597.742095] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12597.745383] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12597.748036] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12597.766416] Lustre: Skipped 2 previous similar messages [12597.788304] Lustre: Skipped 1 previous similar message [12601.312933] Lustre: server umount lustre-MDT0001 complete [12602.313598] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12602.328186] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12608.718094] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12608.795664] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12608.800319] Lustre: Found index 1 for lustre-MDT0001, updating log [12608.835865] Lustre: 358303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12609.077552] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12613.137433] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12614.129038] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12616.197707] LustreError: 370890:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12616.208257] LustreError: 370890:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12616.222106] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12616.227378] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12616.276842] LustreError: 370893:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12616.285473] LustreError: 370893:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12616.289666] LustreError: 370893:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -17 [12616.300771] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.135@tcp [12616.300771] [12919.789570] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.35@tcp (stopping) [12921.315096] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12921.323153] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12921.334229] Lustre: Skipped 1 previous similar message [12924.435725] Lustre: server umount lustre-MDT0001 complete [12924.876262] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.35@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12924.896765] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [12933.350886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12933.444180] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12933.449166] Lustre: Found index 1 for lustre-MDT0001, updating log [12933.502807] Lustre: 358303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12933.767982] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12937.905879] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [12939.262882] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12941.811909] LustreError: 380012:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12941.823755] LustreError: 380012:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12941.834052] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12941.843089] LustreError: 358309:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12941.914858] LustreError: 380015:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12941.928889] LustreError: 380015:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.135@tcp: cfg command failed: rc = -114 [12941.935636] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12941.935636] [13261.286489] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13261.315514] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13261.332908] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13261.342851] Lustre: Skipped 2 previous similar messages [13263.377447] Lustre: server umount lustre-MDT0000 complete [13266.898325] LustreError: 358297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390759 with bad export cookie 8572206564590144569 [13266.901049] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13266.918474] LustreError: 358297:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13266.933397] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13266.936810] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13266.936822] LustreError: 358311:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [13266.962817] Lustre: Skipped 2 previous similar messages [13266.967103] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13266.974480] Lustre: Skipped 2 previous similar messages [13277.154245] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13277.163778] Lustre: Skipped 1 previous similar message [13281.249312] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13281.584474] Lustre: server umount lustre-MDT0001 complete [13286.278617] Lustre: server umount lustre-OST0000 complete [13297.643623] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [13309.997645] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [13347.990762] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 05:27:18 (1787390838) [13356.480695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13356.902911] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13356.967184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13357.101090] Lustre: lustre-MDT0000: new disk, initializing [13357.218556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13357.256213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13362.678567] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [13373.999901] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13374.182213] Lustre: 390995:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13374.265491] LustreError: 391006:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13374.286812] LustreError: 391006:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13374.287962] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13374.302923] Lustre: Skipped 1 previous similar message [13374.414369] Lustre: lustre-MDT0001: new disk, initializing [13374.509890] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13374.549727] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13374.555076] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13379.009635] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [13387.931894] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13392.958865] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13399.809745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13400.261484] Lustre: lustre-OST0000: new disk, initializing [13400.267771] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13400.278565] Lustre: 392966:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13400.389201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13405.759714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13405.772935] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13405.915055] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13406.636949] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_default_debug -1 all [13415.356662] Lustre: DEBUG MARKER: oleg335-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13424.104514] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13424.111488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13424.127427] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13424.736776] Lustre: server umount lustre-MDT0000 complete [13428.500346] LustreError: 390986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787390921 with bad export cookie 8572206564590146522 [13428.501326] LustreError: MGC192.168.203.135@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13428.506328] LustreError: 391807:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13428.519754] LustreError: 390986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13428.824818] Lustre: server umount lustre-MDT0001 complete [13442.728417] Lustre: server umount lustre-OST0000 complete [13472.842885] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing set_hostid [13483.686299] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing load_modules_local [13517.639821] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13221 sec ========= 05:30:08 (1787391008) [13519.871900] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 05:30:10 (1787391010) === [13523.475323] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 05:30:14 (1787391014) === [13561.978946] Lustre: DEBUG MARKER: oleg335-server.virtnet: executing unload_modules_local [13564.718580] Key type lgssc unregistered [13565.092524] LNet: 399331:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13565.101213] LNetError: 399331:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13566.119895] LNet: Removed LNI 192.168.203.135@tcp [13567.137486] Key type .llcrypt unregistered [13567.146898] Key type ._llcrypt unregistered