[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 480295037 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002352] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007027] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008017] pid_max: default: 32768 minimum: 301 [ 0.009147] LSM: Security Framework initializing [ 0.011053] Yama: becoming mindful. [ 0.012052] SELinux: Initializing. [ 0.013103] *** VALIDATE selinux *** [ 0.021913] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028262] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029175] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030122] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031126] *** VALIDATE tmpfs *** [ 0.033411] *** VALIDATE proc *** [ 0.034259] *** VALIDATE cgroup *** [ 0.035011] *** VALIDATE cgroup2 *** [ 0.036299] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037155] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039041] Spectre V2 : User space: Vulnerable [ 0.040010] Speculative Store Bypass: Vulnerable [ 0.043672] debug: unmapping init [mem 0xffffffffb1659000-0xffffffffb1660fff] [ 0.045286] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.046838] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.047032] ... version: 2 [ 0.048014] ... bit width: 48 [ 0.049014] ... generic registers: 4 [ 0.050013] ... value mask: 0000ffffffffffff [ 0.051017] ... max period: 00007fffffffffff [ 0.052018] ... fixed-purpose events: 3 [ 0.053015] ... event mask: 000000070000000f [ 0.054380] rcu: Hierarchical SRCU implementation. [ 0.056545] smp: Bringing up secondary CPUs ... [ 0.057733] x86: Booting SMP configuration: [ 0.058036] .... node #0, CPUs: #1 #2 #3 [ 0.063673] smp: Brought up 1 node, 4 CPUs [ 0.065013] smpboot: Max logical packages: 1 [ 0.066018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.121634] node 0 deferred pages initialised in 53ms [ 0.125172] devtmpfs: initialized [ 0.126191] x86/mm: Memory block size: 128MB [ 0.128273] gcov: version magic: 0x41383552 [ 0.130017] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.132088] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.134332] pinctrl core: initialized pinctrl subsystem [ 0.135120] [ 0.135565] ************************************************************* [ 0.137009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.140013] ** ** [ 0.143014] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.145013] ** ** [ 0.148015] ** This means that this kernel is built to expose internal ** [ 0.150010] ** IOMMU data structures, which may compromise security on ** [ 0.153015] ** your system. ** [ 0.155012] ** ** [ 0.158014] ** If you see this message and you are not debugging the ** [ 0.161013] ** kernel, report this immediately to your vendor! ** [ 0.162012] ** ** [ 0.165014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.167009] ************************************************************* [ 0.168560] NET: Registered protocol family 16 [ 0.170344] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.171042] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.173038] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.176008] cpuidle: using governor menu [ 0.177024] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.178331] PCI: Using configuration type 1 for base access [ 0.180099] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.190070] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.191013] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.194071] cryptd: max_cpu_qlen set to 1000 [ 0.197241] ACPI: Added _OSI(Module Device) [ 0.198014] ACPI: Added _OSI(Processor Device) [ 0.200011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.201010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.205823] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.211464] ACPI: Interpreter enabled [ 0.213055] ACPI: PM: (supports S0 S3 S4 S5) [ 0.214012] ACPI: Using IOAPIC for interrupt routing [ 0.216101] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.219791] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.229418] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.232040] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.234020] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.238076] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.243344] acpiphp: Slot [2] registered [ 0.245083] acpiphp: Slot [5] registered [ 0.246097] acpiphp: Slot [6] registered [ 0.247046] acpiphp: Slot [7] registered [ 0.247981] acpiphp: Slot [8] registered [ 0.248075] acpiphp: Slot [9] registered [ 0.248965] acpiphp: Slot [10] registered [ 0.250066] acpiphp: Slot [3] registered [ 0.251033] acpiphp: Slot [4] registered [ 0.251940] acpiphp: Slot [11] registered [ 0.253060] acpiphp: Slot [12] registered [ 0.254029] acpiphp: Slot [13] registered [ 0.255088] acpiphp: Slot [14] registered [ 0.257153] acpiphp: Slot [15] registered [ 0.258094] acpiphp: Slot [16] registered [ 0.260183] acpiphp: Slot [17] registered [ 0.261086] acpiphp: Slot [18] registered [ 0.262113] acpiphp: Slot [19] registered [ 0.264098] acpiphp: Slot [20] registered [ 0.265159] acpiphp: Slot [21] registered [ 0.267083] acpiphp: Slot [22] registered [ 0.268159] acpiphp: Slot [23] registered [ 0.270123] acpiphp: Slot [24] registered [ 0.272097] acpiphp: Slot [25] registered [ 0.273098] acpiphp: Slot [26] registered [ 0.275130] acpiphp: Slot [27] registered [ 0.276111] acpiphp: Slot [28] registered [ 0.278133] acpiphp: Slot [29] registered [ 0.279115] acpiphp: Slot [30] registered [ 0.281109] acpiphp: Slot [31] registered [ 0.282102] PCI host bridge to bus 0000:00 [ 0.284022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.286034] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.289037] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.292029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.295030] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.298026] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.300197] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.303060] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.306363] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.317781] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.322053] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.325021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.327021] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.330021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.333445] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.335872] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.339043] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.342916] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.348014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.362022] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.368029] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.373000] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.382018] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.393017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.407019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.417751] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.424021] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.430017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.448026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.455440] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.462014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.468018] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.483041] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.493186] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.500017] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.506017] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.520014] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.526554] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.533022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.542016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.564022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.571681] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.579018] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.585016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.600017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.609133] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.611316] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.613277] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.615325] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.617184] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.623177] iommu: Default domain type: Passthrough [ 0.624653] SCSI subsystem initialized [ 0.626141] ACPI: bus type USB registered [ 0.627128] usbcore: registered new interface driver usbfs [ 0.629079] usbcore: registered new interface driver hub [ 0.631096] usbcore: registered new device driver usb [ 0.632205] pps_core: LinuxPPS API ver. 1 registered [ 0.634011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.637055] PTP clock support registered [ 0.639091] EDAC MC: Ver: 3.0.0 [ 0.641110] PCI: Using ACPI for IRQ routing [ 0.642661] NetLabel: Initializing [ 0.644011] NetLabel: domain hash size = 128 [ 0.645009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.647102] NetLabel: unlabeled traffic allowed by default [ 0.649134] vgaarb: loaded [ 0.650231] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.652007] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.661174] clocksource: Switched to clocksource kvm-clock [ 0.755553] VFS: Disk quotas dquot_6.6.0 [ 0.757293] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.759498] *** VALIDATE ramfs *** [ 0.760822] *** VALIDATE hugetlbfs *** [ 0.762632] pnp: PnP ACPI init [ 0.764792] pnp: PnP ACPI: found 6 devices [ 0.790636] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.794020] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.795613] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.797018] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.798445] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.799704] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.801206] NET: Registered protocol family 2 [ 0.802751] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.806082] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.808301] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.812122] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.815212] TCP: Hash tables configured (established 65536 bind 65536) [ 0.818272] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.821397] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.823126] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.825218] NET: Registered protocol family 1 [ 0.827917] RPC: Registered named UNIX socket transport module. [ 0.830153] RPC: Registered udp transport module. [ 0.832051] RPC: Registered tcp transport module. [ 0.833791] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.836192] NET: Registered protocol family 44 [ 0.837797] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.840060] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.842163] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.844707] PCI: CLS 0 bytes, default 64 [ 0.846481] Unpacking initramfs... [ 2.293723] debug: unmapping init [mem 0xffff9ff1bcc54000-0xffff9ff1bffbffff] [ 2.297769] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.300233] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.303336] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.812917] Initialise system trusted keyrings [ 2.814449] Key type blacklist registered [ 2.816160] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.864353] zbud: loaded [ 2.869101] *** VALIDATE nfs *** [ 2.870465] *** VALIDATE nfs4 *** [ 2.872182] pstore: using deflate compression [ 2.875881] Platform Keyring initialized [ 2.980705] NET: Registered protocol family 38 [ 2.982032] Key type asymmetric registered [ 2.983449] Asymmetric key parser 'x509' registered [ 2.985156] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.987988] io scheduler mq-deadline registered [ 2.989076] io scheduler kyber registered [ 2.990156] io scheduler bfq registered [ 2.991508] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.993551] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.996771] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.999936] ACPI: Power Button [PWRF] [ 3.005907] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.014459] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.030255] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.037463] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.055098] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.082234] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.109507] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.114787] Non-volatile memory driver v1.3 [ 3.116627] Linux agpgart interface v0.103 [ 3.148315] virtio_blk virtio1: [vda] 147088 512-byte logical blocks (75.3 MB/71.8 MiB) [ 3.151620] vda: detected capacity change from 0 to 75309056 [ 3.165628] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.169141] vdb: detected capacity change from 0 to 1073741824 [ 3.191476] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.194230] vdc: detected capacity change from 0 to 2621440000 [ 3.210310] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.214109] vdd: detected capacity change from 0 to 2621440000 [ 3.233520] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.236667] vde: detected capacity change from 0 to 4294967296 [ 3.251405] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.255725] vdf: detected capacity change from 0 to 4294967296 [ 3.268157] libphy: Fixed MDIO Bus: probed [ 3.279146] usbcore: registered new interface driver usbserial_generic [ 3.281757] usbserial: USB Serial support registered for generic [ 3.284236] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.288103] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.290131] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.292571] mousedev: PS/2 mouse device common for all mice [ 3.295467] rtc_cmos 00:05: RTC can wake from S4 [ 3.297490] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.301045] rtc_cmos 00:05: registered as rtc0 [ 3.304402] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.308551] intel_pstate: CPU model not supported [ 3.311797] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.314292] hid: raw HID events driver (C) Jiri Kosina [ 3.317388] usbcore: registered new interface driver usbhid [ 3.319092] usbhid: USB HID core driver [ 3.319728] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.320294] drop_monitor: Initializing network drop monitor service [ 3.324662] Initializing XFRM netlink socket [ 3.326890] NET: Registered protocol family 10 [ 3.329429] Segment Routing with IPv6 [ 3.331254] NET: Registered protocol family 17 [ 3.333293] mpls_gso: MPLS GSO support [ 3.338807] RAS: Correctable Errors collector initialized. [ 3.341154] AVX version of gcm_enc/dec engaged. [ 3.343029] AES CTR mode by8 optimization enabled [ 3.417505] sched_clock: Marking stable (3417469765, 0)->(4335727943, -918258178) [ 3.421342] registered taskstats version 1 [ 3.423483] Loading compiled-in X.509 certificates [ 3.425038] zswap: loaded using pool lzo/zbud [ 3.447656] Key type big_key registered [ 3.458049] Key type encrypted registered [ 3.459575] ima: No TPM chip found, activating TPM-bypass! [ 3.461697] ima: Allocated hash algorithm: sha1 [ 3.463387] ima: No architecture policies found [ 3.465269] evm: Initialising EVM extended attributes: [ 3.467403] evm: security.selinux [ 3.468703] evm: security.ima [ 3.469839] evm: security.capability [ 3.471173] evm: HMAC attrs: 0x1 [ 3.473585] rtc_cmos 00:05: setting system clock to 2026-08-27 19:59:54 UTC (1787860794) [ 3.480146] debug: unmapping init [mem 0xffffffffb2603000-0xffffffffb27fffff] [ 3.483533] debug: unmapping init [mem 0xffffffffb1382000-0xffffffffb1658fff] [ 3.492086] Write protecting the kernel read-only data: 28672k [ 3.496184] debug: unmapping init [mem 0xffffffffafa03000-0xffffffffafbfffff] [ 3.499631] debug: unmapping init [mem 0xffffffffb0314000-0xffffffffb03fffff] [ 3.538961] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.549283] systemd[1]: Detected virtualization kvm. [ 3.551346] systemd[1]: Detected architecture x86-64. [ 3.552907] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.579780] systemd[1]: No hostname configured. [ 3.581682] systemd[1]: Set hostname to . [ 3.584126] random: systemd: uninitialized urandom read (16 bytes read) [ 3.586781] systemd[1]: Initializing machine ID from random generator. [ 3.641598] random: ln: uninitialized urandom read (6 bytes read) [ 3.727250] random: systemd: uninitialized urandom read (16 bytes read) [ 3.730028] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.734155] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.739393] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Swap. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.318059] device-mapper: uevent: version 1.0.3 [ 4.320087] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook..[ 4.963428] random: fast init done . [ 4.976805] virtio_net virtio0 ens2: renamed from eth0 [ 5.085827] scsi host0: ata_piix [ 5.113741] scsi host1: ata_piix [ 5.115376] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.117935] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.757880] dracut-initqueue[584]: RTNETLINK answers: File exists [ 9.977069] random: crng init done [ 9.978283] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.344205] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.499945] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.739493] SELinux: Disabled at runtime. [ 11.800565] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.809220] systemd[1]: Detected virtualization kvm. [ 11.811149] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.263927] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.266177] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.270940] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.275314] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.279567] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.287538] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.292932] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ 12.330173] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on initctl Compatibility Named Pipe. Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... Starting Remount Root and Kernel File Systems... [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Mounted /mnt. [ OK ] Started udev Coldplug all Devices. [ 12.776821] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.043762] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.055388] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.284273] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.295520] EDAC sbridge: Ver: 1.1.2 [ 14.673413] Key type dns_resolver registered [ 14.976768] NFS: Registering the id_resolver key type [ 14.978973] Key type id_resolver registered [ 14.980666] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started irqbalance daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg210-server login: [ 42.039333] hrtimer: interrupt took 3322638 ns [ 72.341408] libcfs: loading out-of-tree module taints kernel. [ 72.395375] Key type ._llcrypt registered [ 72.396671] Key type .llcrypt registered [ 72.616518] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 93.045086] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 95.179695] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 95.211782] alg: No test for adler32 (adler32-zlib) [ 96.813165] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 97.912780] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 99.703689] Key type lgssc registered [ 102.200908] Lustre: Echo OBD driver; http://www.lustre.org/ [ 119.266725] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 167.568187] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 183.913951] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 183.982157] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 185.451123] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 185.514511] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 185.636708] Lustre: lustre-MDT0000: new disk, initializing [ 185.712736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 185.732500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 191.339702] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.568774] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 207.703845] Lustre: 6529:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 207.756657] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 207.767991] Lustre: Skipped 1 previous similar message [ 207.886282] Lustre: lustre-MDT0001: new disk, initializing [ 207.969591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 207.990666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 208.003710] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 212.304514] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 217.807450] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 230.941691] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 231.485528] Lustre: lustre-OST0000: new disk, initializing [ 231.490235] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 231.498261] Lustre: 8468:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 231.621416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 240.423927] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 241.234663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 241.248418] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 241.377647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 259.235159] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 259.463959] Lustre: lustre-OST0001: new disk, initializing [ 259.472495] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 259.490088] Lustre: 9545:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 259.573686] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 265.822290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 265.856389] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 265.922761] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 269.265317] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 283.536375] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 292.451064] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 299.548866] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing check_logdir /tmp/testlogs/ [ 308.175223] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing yml_node [ 313.607471] Lustre: DEBUG MARKER: Client: 2.17.57.120 [ 317.678582] Lustre: DEBUG MARKER: MDS: 2.17.57.120 [ 322.316713] Lustre: DEBUG MARKER: OSS: 2.17.57.120 [ 324.636267] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 16:05:14 EDT 2026 [ 347.294447] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 349.025634] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 366.560357] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 366.574678] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 366.593286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 368.612065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 368.612961] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.088266] Lustre: server umount lustre-MDT0000 complete [ 376.677075] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861167 with bad export cookie 4045394405297548467 [ 376.678566] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 376.688338] LustreError: 6521:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 378.862310] LustreError: 6536:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 378.863087] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 378.864444] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 378.864451] Lustre: Skipped 3 previous similar messages [ 378.877832] LustreError: 6536:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 378.921883] Lustre: Skipped 3 previous similar messages [ 381.856614] LustreError: 6538:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 381.857678] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 381.897941] LustreError: 6538:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 381.919687] Lustre: Skipped 2 previous similar messages [ 383.252601] Lustre: server umount lustre-MDT0001 complete [ 388.150688] Lustre: server umount lustre-OST0000 complete [ 393.026518] Lustre: server umount lustre-OST0001 complete [ 401.081976] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 411.460821] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 451.486382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 451.773253] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 451.812654] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 451.888782] Lustre: lustre-MDT0000: new disk, initializing [ 451.977907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 451.997634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 456.974971] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 470.093086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 470.286082] Lustre: 15102:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 470.358587] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 470.362332] Lustre: Skipped 1 previous similar message [ 470.491451] Lustre: lustre-MDT0001: new disk, initializing [ 470.605801] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 470.651578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 470.667231] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 476.000308] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 489.313338] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 496.436876] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 504.975973] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 505.285477] Lustre: lustre-OST0000: new disk, initializing [ 505.288822] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 505.295057] Lustre: 17072:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 505.389962] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 506.778674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 506.794219] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 506.949737] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 512.100198] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 522.353104] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 527.590335] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 527.823957] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 533.225990] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 533.479402] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 537.573776] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 537.576142] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 537.602172] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 537.629810] Lustre: Skipped 1 previous similar message [ 541.944499] Lustre: server umount lustre-OST0000 complete [ 547.297072] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 547.305945] LustreError: Skipped 1 previous similar message [ 547.320938] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 547.344085] Lustre: Skipped 1 previous similar message [ 547.358589] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 552.489690] Lustre: server umount lustre-MDT0000 complete [ 552.928787] LustreError: 15112:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 552.951568] LustreError: 15112:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 556.815579] LustreError: 15094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861347 with bad export cookie 4045394405297550000 [ 556.817709] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 556.838078] LustreError: 15094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 557.147413] Lustre: server umount lustre-MDT0001 complete [ 563.378873] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 16:09:13 (1787861353) [ 571.989148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 572.710592] LustreError: 19336:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 572.851259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 578.029208] LustreError: 19337:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 578.141930] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 587.249900] LustreError: 19337:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 587.273135] LustreError: 19337:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 588.455033] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 594.255239] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 604.553203] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 612.483558] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 620.394750] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 628.771435] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 637.526569] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 652.656076] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 664.037279] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 664.043093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 664.062687] Lustre: Skipped 3 previous similar messages [ 664.073834] Lustre: Skipped 3 previous similar messages [ 666.959760] Lustre: server umount lustre-MDT0000 complete [ 669.156478] LustreError: 19332:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 669.180330] LustreError: 19332:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 671.164378] LustreError: 19317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861462 with bad export cookie 4045394405297550882 [ 671.185084] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 671.189383] LustreError: 19317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 671.824504] Lustre: server umount lustre-MDT0001 complete [ 686.732213] Lustre: server umount lustre-OST0000 complete [ 717.128700] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 730.865600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 731.491721] LustreError: 23782:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 731.583316] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 731.589959] Lustre: Skipped 2 previous similar messages [ 737.487725] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 747.951994] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 754.899605] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 758.682694] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 766.033287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 774.733665] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 785.095157] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 785.353752] Lustre: lustre-OST0001: new disk, initializing [ 785.357785] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 785.369566] Lustre: 26098:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 787.320954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 787.341055] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 787.493025] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 794.304235] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 805.291961] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 809.284609] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 823.270511] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 823.277287] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 823.289552] Lustre: Skipped 2 previous similar messages [ 823.300743] Lustre: Skipped 1 previous similar message [ 829.144786] Lustre: server umount lustre-MDT0000 complete [ 833.320569] LustreError: 23763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861624 with bad export cookie 4045394405297551729 [ 833.333877] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 833.334478] LustreError: 23763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 833.513797] LustreError: 23775:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 833.536876] LustreError: 23775:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 833.831401] Lustre: server umount lustre-MDT0001 complete [ 838.824753] Lustre: server umount lustre-OST0000 complete [ 843.470257] Lustre: server umount lustre-OST0001 complete [ 860.878105] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 872.057395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 872.686442] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 872.704699] Lustre: Skipped 3 previous similar messages [ 878.310437] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 888.112502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 892.562909] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 895.580332] Lustre: 30261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 904.174500] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 915.054166] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 925.150500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 931.637946] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 939.226649] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 942.868741] Lustre: 32141:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 950.244809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 950.253085] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 950.263141] Lustre: Skipped 3 previous similar messages [ 950.270902] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 950.281880] Lustre: Skipped 8 previous similar messages [ 955.068308] Lustre: server umount lustre-MDT0000 complete [ 959.004939] LustreError: 29095:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861750 with bad export cookie 4045394405297553017 [ 959.033236] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 959.400135] Lustre: server umount lustre-MDT0001 complete [ 973.141275] Lustre: server umount lustre-OST0000 complete [ 986.509060] Lustre: server umount lustre-OST0001 complete [ 996.754285] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 16:16:27 (1787861787) [ 1031.554837] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1044.555600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1045.035261] LustreError: 34947:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1045.050880] LustreError: 34947:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 13 previous similar messages [ 1050.893923] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1060.690864] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1066.964585] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1071.191214] Lustre: 36097:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1079.164044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1087.244834] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1096.137425] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1104.020263] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1112.421423] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1116.322940] Lustre: 37975:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1122.272631] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1122.283692] Lustre: Skipped 3 previous similar messages [ 1122.292058] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1122.294701] Lustre: Skipped 7 previous similar messages [ 1128.703342] Lustre: server umount lustre-MDT0000 complete [ 1132.256925] LustreError: 34929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787861923 with bad export cookie 4045394405297554130 [ 1132.259829] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1132.271061] LustreError: 34929:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1132.490909] Lustre: server umount lustre-MDT0001 complete [ 1145.421840] Lustre: server umount lustre-OST0000 complete [ 1148.898481] Lustre: 3664:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787861923/real 1787861923] req@ffff9ff20b28a940 x1874708018863104/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787861939 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1151.509896] Lustre: server umount lustre-OST0001 complete [ 1164.376727] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 16:19:14 (1787861954) [ 1184.932796] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1198.784457] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1199.675401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1199.690398] Lustre: Skipped 7 previous similar messages [ 1204.995930] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1215.051989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1220.915372] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1224.473432] Lustre: 41268:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1231.747034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1239.486348] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1249.631951] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1257.363734] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1266.393586] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1270.440864] Lustre: 43145:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1271.556450] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1271.564720] Lustre: Skipped 1 previous similar message [ 1275.873580] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1275.906702] Lustre: Skipped 6 previous similar messages [ 1281.067765] Lustre: server umount lustre-MDT0000 complete [ 1285.032646] LustreError: 40100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787862076 with bad export cookie 4045394405297555236 [ 1285.036729] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1285.036987] LustreError: 40100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1291.461804] Lustre: server umount lustre-MDT0001 complete [ 1302.686425] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1310.693688] LustreError: 3660:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9ff22dda52c0 x1874708018915328/t0(0) o250->MGC192.168.202.110@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1310.976643] LustreError: 41632:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1310.997564] LustreError: 41632:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 25 previous similar messages [ 1318.964882] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1320.449184] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1329.647973] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1335.794652] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1335.813941] Lustre: Skipped 1 previous similar message [ 1336.847234] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1347.371095] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1349.217804] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1357.543536] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1359.832718] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1365.988266] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1372.207428] Lustre: server umount lustre-MDT0000 complete [ 1376.379524] LustreError: 41651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787862167 with bad export cookie 4045394405297556363 [ 1376.390025] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1376.391881] LustreError: 41651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1376.966369] Lustre: server umount lustre-MDT0001 complete [ 1384.229876] Lustre: server umount lustre-OST0000 complete [ 1388.976808] Lustre: server umount lustre-OST0001 complete [ 1398.644440] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 16:23:09 (1787862189) [ 1418.300129] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1430.953782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1436.883111] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1448.544236] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1455.348528] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1458.740888] Lustre: 48907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1467.251180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1476.901690] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1487.014532] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1494.518380] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1505.572866] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1511.843199] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1528.818516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1528.828685] Lustre: Skipped 26 previous similar messages [ 1532.767826] Lustre: server umount lustre-MDT0000 complete [ 1536.918357] LustreError: 47733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787862327 with bad export cookie 4045394405297557161 [ 1536.948357] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1536.953704] LustreError: 47733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1537.249100] Lustre: server umount lustre-MDT0001 complete [ 1552.263532] Lustre: server umount lustre-OST0000 complete [ 1555.231175] Lustre: 3664:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787862330/real 1787862330] req@ffff9ff229d27c00 x1874708019020032/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787862346 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1555.262073] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1555.280018] Lustre: Skipped 13 previous similar messages [ 1556.894559] Lustre: server umount lustre-OST0001 complete [ 1569.445780] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 16:25:59 (1787862359) [ 1573.564777] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1575.472230] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 16:26:06 (1787862366) [ 1589.335757] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 1600.940745] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1627.676246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1628.179478] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1628.366444] Lustre: lustre-MDT0000: new disk, initializing [ 1628.565405] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1634.117623] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1644.955566] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1653.702931] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1653.970653] Lustre: lustre-OST0000: new disk, initializing [ 1653.975820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1653.981736] Lustre: Skipped 1 previous similar message [ 1653.986925] Lustre: 54987:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1655.205165] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1655.218632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1655.260700] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1661.013036] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1672.171626] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1674.970203] Lustre: DEBUG MARKER: create test files [ 1722.086061] Lustre: server umount lustre-OST0000 complete [ 1727.837034] Lustre: server umount lustre-MDT0000 complete [ 1741.125335] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 1745.076476] Key type lgssc unregistered [ 1745.503024] LNet: 56628:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1745.513319] LNetError: 56628:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1745.543058] LNet: Removed LNI 192.168.202.110@tcp [ 1747.334145] Key type .llcrypt unregistered [ 1747.335806] Key type ._llcrypt unregistered [ 1787.293314] Key type ._llcrypt registered [ 1787.301666] Key type .llcrypt registered [ 1787.448332] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1789.555342] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1789.636426] alg: No test for adler32 (adler32-zlib) [ 1790.668424] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 1791.084424] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 1792.847246] Key type lgssc registered [ 1794.140393] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1803.287901] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1803.349916] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1804.920488] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1810.470535] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1820.379557] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1828.603387] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1828.978775] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1829.995456] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1830.039977] Lustre: 59366:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1830.043488] Lustre: 59366:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1830.047482] Lustre: 59366:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1830.052020] Lustre: 59366:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1830.056380] Lustre: 59366:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1830.060615] Lustre: 59366:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1835.933483] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1845.833128] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1852.717900] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1859.158757] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1871.328695] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1871.338465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1871.355133] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1876.454077] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1881.055176] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1881.231080] Lustre: server umount lustre-OST0000 complete [ 1886.269906] Lustre: server umount lustre-MDT0000 complete [ 1901.480869] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 1905.429702] Key type lgssc unregistered [ 1905.785769] LNet: 60980:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1905.797364] LNetError: 60980:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1905.813812] LNet: Removed LNI 192.168.202.110@tcp [ 1906.917911] Key type .llcrypt unregistered [ 1906.919361] Key type ._llcrypt unregistered [ 1949.659812] Key type ._llcrypt registered [ 1949.671214] Key type .llcrypt registered [ 1950.035775] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 1952.429829] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1952.481429] alg: No test for adler32 (adler32-zlib) [ 1953.902886] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 1954.292533] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 1956.271244] Key type lgssc registered [ 1958.199697] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1969.327489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1969.364291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1970.992478] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1976.282565] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 1987.744741] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1996.600456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1996.836456] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2004.386731] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2013.386085] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2015.225818] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2015.429494] Lustre: 64487:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 2015.437667] Lustre: 64487:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 2015.448201] Lustre: 64487:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2015.457985] Lustre: 64487:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2015.474313] Lustre: 64487:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2015.486349] Lustre: 64487:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2024.384171] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2035.683951] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2035.685599] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2035.707388] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2040.847797] Lustre: server umount lustre-OST0000 complete [ 2044.958777] Lustre: server umount lustre-MDT0000 complete [ 2056.278869] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 2059.813840] Key type lgssc unregistered [ 2060.130499] LNet: 65392:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2060.138893] LNetError: 65392:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2060.152681] LNet: Removed LNI 192.168.202.110@tcp [ 2061.131151] Key type .llcrypt unregistered [ 2061.133315] Key type ._llcrypt unregistered [ 2079.731092] Key type ._llcrypt registered [ 2079.733564] Key type .llcrypt registered [ 2079.842376] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 2095.226329] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 2096.689032] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2096.715976] alg: No test for adler32 (adler32-zlib) [ 2097.790650] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 2097.998534] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 2099.778616] Key type lgssc registered [ 2101.183801] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2143.099306] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2143.153640] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2144.649462] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2144.731636] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2144.905424] Lustre: lustre-MDT0000: new disk, initializing [ 2145.066456] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2145.094308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2150.977144] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2164.537040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2164.769785] Lustre: 68551:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2164.830238] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2164.833933] Lustre: Skipped 1 previous similar message [ 2164.910389] Lustre: lustre-MDT0001: new disk, initializing [ 2164.991430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2165.050229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2165.067471] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2170.616663] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2183.008342] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2191.718811] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2203.182257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2203.608441] Lustre: lustre-OST0000: new disk, initializing [ 2203.611917] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2203.620383] Lustre: 70520:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2203.730534] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2210.457436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2210.461477] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2210.587901] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2212.679576] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2225.827690] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2233.154143] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2233.457680] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2239.605139] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2239.928708] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2246.631842] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2246.633108] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2246.648184] LustreError: Skipped 1 previous similar message [ 2246.658479] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2246.678154] Lustre: Skipped 1 previous similar message [ 2248.780216] Lustre: server umount lustre-OST0000 complete [ 2256.864396] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2256.868116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2256.897380] Lustre: Skipped 1 previous similar message [ 2256.919537] Lustre: Skipped 2 previous similar messages [ 2259.778808] Lustre: server umount lustre-MDT0000 complete [ 2264.967282] LustreError: 68542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787863055 with bad export cookie 8735637090416759797 [ 2264.970683] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2264.982707] LustreError: 68542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2265.616966] Lustre: server umount lustre-MDT0001 complete [ 2275.439438] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 16:37:45 (1787863065) [ 2288.259443] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 2339.144813] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 2351.399611] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 2398.167492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2398.631683] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2398.708833] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2398.957428] Lustre: lustre-MDT0000: new disk, initializing [ 2399.165126] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2399.187200] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2405.094654] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2418.395730] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2418.534928] Lustre: 76550:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2418.593672] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2418.598656] Lustre: Skipped 1 previous similar message [ 2418.751183] Lustre: lustre-MDT0001: new disk, initializing [ 2418.916097] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2418.967663] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2418.995307] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2425.127527] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2436.082227] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2442.981866] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2452.533703] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2452.963522] Lustre: lustre-OST0000: new disk, initializing [ 2452.970760] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2452.986520] Lustre: 78519:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2453.116045] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2461.682569] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2462.349045] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2462.372811] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2462.566791] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2473.771211] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2481.406823] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2481.788895] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2487.540870] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2487.838812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2493.410828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2493.421174] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2493.441367] Lustre: Skipped 1 previous similar message [ 2493.449429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2493.489492] Lustre: Skipped 1 previous similar message [ 2496.926166] Lustre: server umount lustre-OST0000 complete [ 2503.668166] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2503.679793] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2503.693169] Lustre: Skipped 1 previous similar message [ 2503.717376] Lustre: Skipped 1 previous similar message [ 2508.796754] LustreError: 76561:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2508.828213] LustreError: 76561:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2508.904826] Lustre: server umount lustre-MDT0000 complete [ 2513.765439] LustreError: 76542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787863304 with bad export cookie 8735637090416760672 [ 2513.766070] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2513.793741] LustreError: 76542:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2514.244026] Lustre: server umount lustre-MDT0001 complete [ 2525.481808] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 16:41:55 (1787863315) [ 2533.530954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2533.848859] LustreError: 80935:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2533.865655] LustreError: 80935:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2533.896215] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2538.994784] LustreError: 80936:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2541.371766] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2543.155529] LustreError: 80935:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2548.196519] LustreError: 80936:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2554.490329] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2560.598065] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2569.345140] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2576.294629] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2585.504305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2586.105942] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2586.109597] Lustre: Skipped 1 previous similar message [ 2594.446347] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2604.755117] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2609.349223] Lustre: 83609:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 382a3674-f1f8-4d86-9d7e-9b2f3fac8447 at adminstrative request [ 2613.747489] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2613.763474] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2623.988667] Lustre: MGS: Client 382a3674-f1f8-4d86-9d7e-9b2f3fac8447 (at 0@lo) reconnecting [ 2624.011116] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x793b37a67d049ad9 to 0x793b37a67d049d79 [ 2624.027172] Lustre: MGC192.168.202.110@tcp: Connection restored to 0@lo (at 0@lo) [ 2647.226169] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2647.467438] Lustre: lustre-OST0001: new disk, initializing [ 2647.472754] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2647.479674] Lustre: 84003:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2654.094706] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2654.889491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2654.908562] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2654.978339] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2664.707982] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2670.565240] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2670.574289] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2670.581603] LustreError: Skipped 2 previous similar messages [ 2670.593313] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2670.594488] Lustre: Skipped 1 previous similar message [ 2670.614589] Lustre: Skipped 1 previous similar message [ 2680.802066] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2680.808555] Lustre: Skipped 3 previous similar messages [ 2682.335187] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2682.439229] Lustre: server umount lustre-OST0001 complete [ 2685.939179] LustreError: 84595:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2685.959637] LustreError: 84595:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 2687.457148] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2687.460244] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2692.333732] Lustre: server umount lustre-OST0000 complete [ 2701.280384] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2701.285549] Lustre: Skipped 7 previous similar messages [ 2702.392773] Lustre: server umount lustre-MDT0000 complete [ 2706.078329] LustreError: 80915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787863497 with bad export cookie 8735637090416762233 [ 2706.084557] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2706.087596] LustreError: 80915:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2706.501663] Lustre: server umount lustre-MDT0001 complete [ 2716.095457] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 16:45:06 (1787863506) [ 2717.804217] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2719.844530] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 16:45:10 (1787863510) [ 2721.413626] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2723.127635] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 16:45:13 (1787863513) [ 2724.869075] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2726.681473] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 16:45:17 (1787863517) [ 2728.357441] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2730.196553] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 16:45:20 (1787863520) [ 2731.999958] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2738.785524] loop: module loaded [ 2738.804454] loop0: detected capacity change from 0 to 4294967296 [ 2738.908020] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2740.516345] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2744.018992] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2748.203698] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2781.929433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2782.006053] LustreError: 88132:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2782.019835] LustreError: 88132:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2782.029125] LustreError: 88132:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2782.047464] LustreError: 88132:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2806.565819] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 2816.971496] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 2854.807218] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2855.203577] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2855.258962] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2855.377356] Lustre: lustre-MDT0000: new disk, initializing [ 2855.481991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2855.487361] Lustre: Skipped 1 previous similar message [ 2855.508591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2860.489409] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2871.994860] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2872.109134] Lustre: 90997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2872.303752] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2877.117373] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2886.803036] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2893.205407] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2900.916084] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2901.196691] Lustre: 92967:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2907.046458] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2910.272779] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2910.381059] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2918.118683] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2923.476463] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2923.712706] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2928.629289] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2928.921963] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2931.169836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2931.185350] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2931.184930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2931.201350] Lustre: Skipped 4 previous similar messages [ 2931.235411] LustreError: Skipped 1 previous similar message [ 2937.324426] Lustre: server umount lustre-OST0000 complete [ 2943.967928] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2947.864040] Lustre: server umount lustre-MDT0000 complete [ 2951.655253] LustreError: 92115:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2951.692487] LustreError: 92115:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2953.312570] LustreError: 90989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787863744 with bad export cookie 8735637090416762849 [ 2953.332952] LustreError: 90989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2953.346562] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2954.001205] Lustre: server umount lustre-MDT0001 complete [ 2965.264525] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 16:49:15 (1787863755) [ 2975.525120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2982.850360] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 2986.976309] LustreError: 95379:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2986.981185] LustreError: 95379:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 2991.835187] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2992.101705] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2992.109834] Lustre: Skipped 3 previous similar messages [ 2997.050967] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3005.569644] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3013.141884] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3021.167959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3027.220090] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3034.931697] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3039.952338] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3040.338944] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3046.627740] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3046.919596] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3054.422258] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3054.592362] Lustre: lustre-OST0001: new disk, initializing [ 3054.597396] Lustre: Skipped 2 previous similar messages [ 3054.603286] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3054.609575] Lustre: Skipped 3 previous similar messages [ 3054.618260] Lustre: 98757:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3060.411616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3060.432298] Lustre: Skipped 2 previous similar messages [ 3060.443531] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3060.649596] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3062.516719] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3075.890416] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3081.750428] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3082.086553] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3089.014700] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3089.408236] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3111.795158] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3140.294163] Lustre: Failing over lustre-MDT0000 [ 3140.581862] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3140.588853] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3140.606222] Lustre: Skipped 2 previous similar messages [ 3140.614988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3140.625512] Lustre: Skipped 6 previous similar messages [ 3140.848258] Lustre: server umount lustre-MDT0000 complete [ 3142.642797] LustreError: 95379:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3142.657419] LustreError: 95379:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 3157.987375] Lustre: 66774:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787863933/real 1787863933] req@0000000098156b74 x1874710116991232/t0(0) o400->MGC192.168.202.110@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787863949 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3158.017306] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3166.986034] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3166.989385] LDISKFS-fs (dm-0): recovery complete [ 3167.011868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3168.160463] LustreError: 101571:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 3168.172692] LustreError: 101571:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@00000000cdebe4d3 x1874710116996864/t0(0) o250->MGC192.168.202.110@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787863959 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3168.202327] LustreError: 101571:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 3168.493249] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3170.096297] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3170.106036] Lustre: 95375:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3173.327968] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3173.867968] Lustre: 95379:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3173.870758] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3173.887923] Lustre: 95379:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 3173.963324] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3174.279288] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3174.615356] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3174.625921] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3174.959150] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3174.964730] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3174.977196] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3174.986043] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 3175.625727] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3175.633860] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 3176.279493] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3176.288844] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 3 previous similar messages [ 3177.257600] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3177.269631] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 6 previous similar messages [ 3177.918334] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3177.939537] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 3178.575982] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3178.582743] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 3181.433299] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3181.452866] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 3182.097363] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3182.102904] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3182.735135] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3182.741587] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3189.706223] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3189.720189] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 25 previous similar messages [ 3190.368947] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3190.380111] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 3191.031323] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3191.042480] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 3206.058997] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3206.071010] Lustre: 101603:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 50 previous similar messages [ 3206.396092] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3206.407476] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 49 previous similar messages [ 3207.048386] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3207.053299] LustreError: 101603:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 49 previous similar messages [ 3230.192773] Lustre: 101603:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3230.200875] Lustre: 101603:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3230.208386] Lustre: 101603:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 575ea36b-1759-445d-8575-e6b2da8d321d@192.168.202.10@tcp [ 3230.216676] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3230.229585] Lustre: 101603:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000af0de01e x1874710101037440/t0(8589934772) o35->575ea36b-1759-445d-8575-e6b2da8d321d@192.168.202.10@tcp:626/0 lens 392/0 e 0 to 0 dl 1787864031 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3230.295835] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3230.301166] Lustre: lustre-MDT0000: Denying connection for new client 575ea36b-1759-445d-8575-e6b2da8d321d (at 192.168.202.10@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3230.365286] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3230.425114] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3230.436767] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3230.441751] Lustre: Skipped 2 previous similar messages [ 3230.482395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3230.483401] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3248.102158] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3248.114531] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3248.131562] Lustre: Skipped 3 previous similar messages [ 3253.051811] Lustre: server umount lustre-OST0000 complete [ 3263.364743] Lustre: server umount lustre-OST0001 complete [ 3271.140857] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3271.154227] Lustre: Skipped 6 previous similar messages [ 3274.378499] Lustre: server umount lustre-MDT0000 complete [ 3276.258250] LustreError: 96373:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3276.276906] LustreError: 96373:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 44 previous similar messages [ 3278.677209] LustreError: 95359:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864069 with bad export cookie 8735637090416814061 [ 3278.692972] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3278.979596] Lustre: server umount lustre-MDT0001 complete [ 3288.785744] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 16:54:39 (1787864079) [ 3298.008823] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3298.023202] Lustre: *** cfs_fail_loc=197, val=0*** [ 3313.759690] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3318.880182] LustreError: 103586:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.110@tcp: failed processing log, type 4: rc = -110 [ 3324.110737] Lustre: *** cfs_fail_loc=197, val=0*** [ 3324.120840] Lustre: Skipped 65 previous similar messages [ 3344.671391] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3344.677914] Lustre: Skipped 3 previous similar messages [ 3350.826862] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3360.243903] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3362.518469] Lustre: server umount lustre-OST0000 complete [ 3373.686694] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 16:56:03 (1787864163) [ 3397.190351] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3403.386884] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3414.924641] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3420.262642] LustreError: 105241:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3420.295450] LustreError: 105241:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3420.305804] LustreError: 105241:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 22, retries 0, failed: rc = -5 [ 3420.891209] Lustre: server umount lustre-MDT0000 complete [ 3460.062636] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 3472.377685] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 3518.665281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3519.071497] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3519.116601] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3519.295333] Lustre: lustre-MDT0000: new disk, initializing [ 3519.436251] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3524.305234] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3536.086171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3536.262388] Lustre: 109416:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3536.748811] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3543.042852] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3554.551287] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3561.635801] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3570.337754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3570.677496] Lustre: 111385:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3578.833990] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3579.751213] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3591.111963] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3596.576605] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3596.854988] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3602.782334] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3603.064302] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3605.489550] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3605.494321] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3605.502232] LustreError: Skipped 2 previous similar messages [ 3605.508098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3605.533849] Lustre: Skipped 6 previous similar messages [ 3611.577878] Lustre: server umount lustre-OST0000 complete [ 3622.730977] Lustre: server umount lustre-MDT0000 complete [ 3625.952786] LustreError: 109430:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3625.978257] LustreError: 109430:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [ 3627.140292] LustreError: 109411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864418 with bad export cookie 8735637090416824680 [ 3627.150269] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3627.151362] LustreError: 109411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3627.448891] Lustre: server umount lustre-MDT0001 complete [ 3637.362646] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 17:00:27 (1787864427) [ 3647.019785] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 3650.062566] Key type lgssc unregistered [ 3650.496614] LNet: 113876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3650.503288] LNetError: 113876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3650.533763] LNet: Removed LNI 192.168.202.110@tcp [ 3651.901828] Key type .llcrypt unregistered [ 3651.904618] Key type ._llcrypt unregistered [ 3675.483584] Key type ._llcrypt registered [ 3675.488715] Key type .llcrypt registered [ 3675.706819] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 3692.989900] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 3693.891814] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3693.931930] alg: No test for adler32 (adler32-zlib) [ 3695.448585] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 3695.775223] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 3697.423440] Key type lgssc registered [ 3699.282934] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3746.924924] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3746.973834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3748.449884] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3748.545467] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3748.642489] Lustre: lustre-MDT0000: new disk, initializing [ 3748.687265] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3748.696467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3752.872721] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3763.808833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3764.065490] Lustre: lustre-OST0000: new disk, initializing [ 3764.068677] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3764.072651] Lustre: Skipped 1 previous similar message [ 3764.080683] Lustre: 118065:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3764.154639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3770.534697] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3770.552907] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3770.597308] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3771.644943] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3782.132369] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3782.314515] Lustre: lustre-OST0001: new disk, initializing [ 3782.318209] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3782.323896] Lustre: 118927:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3782.395932] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3789.373567] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 3791.493490] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3791.501977] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3791.539320] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3801.571135] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3801.577182] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3801.598825] Lustre: Skipped 1 previous similar message [ 3805.332340] Lustre: server umount lustre-MDT0000 complete [ 3814.235924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3819.209236] LustreError: 117251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864610 with bad export cookie 544654792443219813 [ 3819.213559] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3819.215246] LustreError: 117251:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3819.256509] Lustre: server umount lustre-OST0000 complete [ 3824.270684] Lustre: server umount lustre-OST0001 complete [ 3833.567987] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 17:03:44 (1787864624) [ 3859.611948] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 17:04:09 (1787864649) [ 3870.168681] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 3899.962313] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 17:04:50 (1787864690) [ 3946.087995] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 3956.226513] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 3999.004587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3999.246379] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3999.298786] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3999.456032] Lustre: lustre-MDT0000: new disk, initializing [ 3999.539217] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3999.567625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4004.483095] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4014.845939] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4015.019399] Lustre: 126684:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4015.120472] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4015.124868] Lustre: Skipped 1 previous similar message [ 4015.218569] Lustre: lustre-MDT0001: new disk, initializing [ 4015.341061] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4015.393181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4015.404321] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4019.736546] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4029.253801] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4034.838381] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4041.626070] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4041.908842] Lustre: lustre-OST0000: new disk, initializing [ 4041.914174] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4041.920870] Lustre: 128654:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4042.019477] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4048.327729] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4049.972210] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4049.979183] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4050.086994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4057.643847] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4062.106730] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4062.354789] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4067.299839] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4067.652987] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4070.897287] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4070.897775] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4070.902247] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4070.902257] Lustre: Skipped 1 previous similar message [ 4070.905098] LustreError: Skipped 1 previous similar message [ 4070.926229] Lustre: Skipped 1 previous similar message [ 4076.012720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4076.023200] Lustre: Skipped 2 previous similar messages [ 4079.017496] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4079.024151] Lustre: Skipped 1 previous similar message [ 4084.191171] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4084.195700] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4084.207721] Lustre: Skipped 1 previous similar message [ 4084.388926] Lustre: server umount lustre-OST0000 complete [ 4089.315387] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4094.434870] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4094.445361] Lustre: Skipped 3 previous similar messages [ 4094.787833] Lustre: server umount lustre-MDT0000 complete [ 4099.229128] LustreError: 127479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787864890 with bad export cookie 544654792443221038 [ 4099.230338] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4099.237602] LustreError: 127479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4099.724130] Lustre: server umount lustre-MDT0001 complete [ 4109.636915] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 17:08:19 (1787864899) [ 4116.483982] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4117.039823] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4121.476511] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4122.083851] LustreError: 131068:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4122.126276] LustreError: 131068:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 4127.220538] LustreError: 131069:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4132.321830] LustreError: 131068:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4132.941286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4139.063649] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4148.193214] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4155.278577] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4163.646872] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4163.941879] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4163.949659] Lustre: Skipped 1 previous similar message [ 4171.177862] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4183.619189] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4193.882427] Lustre: *** cfs_fail_loc=159, val=0*** [ 4210.546218] Lustre: lustre-MDT0000: Client 91d8e31b-4695-4d4d-b355-29d76d46b63e (at 192.168.202.10@tcp) reconnecting [ 4212.200750] Lustre: *** cfs_fail_loc=159, val=0*** [ 4212.204415] Lustre: Skipped 5 previous similar messages [ 4228.436780] Lustre: lustre-MDT0000: Client 91d8e31b-4695-4d4d-b355-29d76d46b63e (at 192.168.202.10@tcp) reconnecting [ 4234.737025] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4234.746537] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4234.753855] Lustre: Skipped 1 previous similar message [ 4234.762665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4234.775287] Lustre: Skipped 1 previous similar message [ 4235.599727] Lustre: server umount lustre-OST0000 complete [ 4239.842735] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4245.936612] Lustre: server umount lustre-MDT0000 complete [ 4250.084845] LustreError: 133733:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4250.091222] LustreError: 131050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865041 with bad export cookie 544654792443221920 [ 4250.097301] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4250.119862] LustreError: 133733:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4250.192488] LustreError: 131050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4250.747361] Lustre: server umount lustre-MDT0001 complete [ 4261.061509] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 4263.473366] Key type lgssc unregistered [ 4263.802468] LNet: 134851:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4263.811978] LNetError: 134851:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4263.846467] LNet: Removed LNI 192.168.202.110@tcp [ 4264.680145] Key type .llcrypt unregistered [ 4264.682067] Key type ._llcrypt unregistered [ 4286.816339] Key type ._llcrypt registered [ 4286.820505] Key type .llcrypt registered [ 4286.940199] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 4288.316469] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4288.365247] alg: No test for adler32 (adler32-zlib) [ 4289.518418] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 4289.800386] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 4291.442180] Key type lgssc registered [ 4292.623809] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4300.036805] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4300.067173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4301.604556] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4306.174722] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4311.019491] LustreError: 136268:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4311.043386] LustreError: 136268:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4314.829895] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4315.274711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4320.019731] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4328.024989] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4333.290688] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4339.626254] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4340.139892] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4346.289928] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4346.831934] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4354.379782] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4363.897486] Lustre: *** cfs_fail_loc=159, val=0*** [ 4380.480341] Lustre: lustre-MDT0000: Client e6e44049-ce98-4be5-9704-80bc2be55d54 (at 192.168.202.10@tcp) reconnecting [ 4383.086727] Lustre: *** cfs_fail_loc=159, val=0*** [ 4383.103345] Lustre: Skipped 2 previous similar messages [ 4399.482199] Lustre: lustre-MDT0001: Client e6e44049-ce98-4be5-9704-80bc2be55d54 (at 192.168.202.10@tcp) reconnecting [ 4401.101625] Lustre: *** cfs_fail_loc=159, val=0*** [ 4401.103447] Lustre: Skipped 1 previous similar message [ 4417.365282] Lustre: lustre-MDT0001: Client e6e44049-ce98-4be5-9704-80bc2be55d54 (at 192.168.202.10@tcp) reconnecting [ 4421.893402] Lustre: 139309:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4424.589041] Lustre: *** cfs_fail_loc=159, val=0*** [ 4424.594417] Lustre: Skipped 4 previous similar messages [ 4439.914909] Lustre: lustre-MDT0000: Client d232c539-735c-4427-8a6e-25626acc4215 (at 192.168.202.10@tcp) reconnecting [ 4441.735200] Lustre: *** cfs_fail_loc=159, val=0*** [ 4441.741617] Lustre: Skipped 15 previous similar messages [ 4457.297228] Lustre: lustre-MDT0000: Client d232c539-735c-4427-8a6e-25626acc4215 (at 192.168.202.10@tcp) reconnecting [ 4459.019996] Lustre: 139603:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4479.465705] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4479.470450] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4479.481618] LustreError: Skipped 1 previous similar message [ 4479.505516] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4479.523869] Lustre: Skipped 1 previous similar message [ 4484.293428] Lustre: server umount lustre-OST0000 complete [ 4489.184370] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4489.196124] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4489.209898] Lustre: Skipped 1 previous similar message [ 4489.214712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4494.817504] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4494.824947] Lustre: Skipped 2 previous similar messages [ 4495.051777] Lustre: server umount lustre-MDT0000 complete [ 4498.978393] LustreError: 136249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865289 with bad export cookie 12717148447579895077 [ 4498.986357] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4498.991201] LustreError: 136249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4499.832868] Lustre: server umount lustre-MDT0001 complete [ 4511.908396] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 4514.965725] Key type lgssc unregistered [ 4515.368379] LNet: 140689:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4515.380459] LNetError: 140689:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4515.405480] LNet: Removed LNI 192.168.202.110@tcp [ 4516.618402] Key type .llcrypt unregistered [ 4516.621173] Key type ._llcrypt unregistered [ 4542.117661] Key type ._llcrypt registered [ 4542.120502] Key type .llcrypt registered [ 4542.224723] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 4543.548712] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4543.808111] alg: No test for adler32 (adler32-zlib) [ 4544.810977] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 4545.053945] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 4546.759196] Key type lgssc registered [ 4548.331259] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4556.315293] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4556.339929] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4558.096196] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4564.000503] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4568.549494] LustreError: 142105:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4568.560890] LustreError: 142105:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4573.664513] LustreError: 142104:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4574.976869] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4575.434488] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4581.562431] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4591.494243] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4597.772749] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4607.158177] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4607.454808] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4616.860571] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4620.796959] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4625.980892] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4628.998801] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4636.810883] Lustre: 144691:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4640.296761] Lustre: 144739:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4642.640438] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4643.556870] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4649.438580] Lustre: 144888:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4650.719704] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4651.915727] Lustre: 144986:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4659.682376] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4659.695239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4659.722183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4659.736266] Lustre: Skipped 1 previous similar message [ 4659.757831] Lustre: Skipped 1 previous similar message [ 4662.964870] Lustre: server umount lustre-OST0000 complete [ 4669.922106] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4669.923760] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4669.946243] Lustre: Skipped 1 previous similar message [ 4669.970040] Lustre: Skipped 1 previous similar message [ 4674.828456] Lustre: server umount lustre-MDT0000 complete [ 4675.040992] LustreError: 143045:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4675.057111] LustreError: 143045:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4679.136693] LustreError: 142104:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4679.148178] LustreError: 142104:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4679.186224] LustreError: 144790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865470 with bad export cookie 312404530669496879 [ 4679.202462] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4679.206145] LustreError: 144790:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4679.698937] Lustre: server umount lustre-MDT0001 complete [ 4693.019856] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 4696.721332] Key type lgssc unregistered [ 4697.134721] LNet: 146050:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4697.139844] LNetError: 146050:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4698.216797] LNet: Removed LNI 192.168.202.110@tcp [ 4698.910350] Key type .llcrypt unregistered [ 4698.912324] Key type ._llcrypt unregistered [ 4725.300420] Key type ._llcrypt registered [ 4725.304057] Key type .llcrypt registered [ 4725.377638] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 4727.095477] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4727.359049] alg: No test for adler32 (adler32-zlib) [ 4729.100406] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 4729.692991] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 4731.625055] Key type lgssc registered [ 4733.693585] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4744.516856] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4744.561561] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4746.329577] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4751.718966] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4756.457431] LustreError: 147468:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4756.481145] LustreError: 147468:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4761.569921] LustreError: 147469:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4762.289221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4762.673400] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4767.246839] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4774.893431] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4781.043994] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4789.858867] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4790.299922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4791.393185] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4797.520904] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4806.995817] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4807.836224] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4813.159349] Lustre: *** cfs_fail_loc=159, val=0*** [ 4829.548787] Lustre: lustre-MDT0000: Client 4c63aaf9-1118-4030-83c3-9898a1bff945 (at 192.168.202.10@tcp) reconnecting [ 4847.590422] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4847.595409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4847.601549] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4849.123149] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4849.142053] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4852.304954] Lustre: server umount lustre-OST0000 complete [ 4859.365527] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4859.371889] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4859.383611] Lustre: Skipped 2 previous similar messages [ 4864.484132] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4864.495818] Lustre: Skipped 1 previous similar message [ 4869.601687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4869.614180] Lustre: Skipped 1 previous similar message [ 4871.135868] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4871.350945] Lustre: server umount lustre-MDT0000 complete [ 4874.721634] LustreError: 147468:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4874.730542] LustreError: 147468:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4875.691118] LustreError: 147449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865666 with bad export cookie 537339345296747587 [ 4875.697523] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4875.704586] LustreError: 147449:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4876.172471] Lustre: server umount lustre-MDT0001 complete [ 4890.733582] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 4894.156569] Key type lgssc unregistered [ 4894.470861] LNet: 151145:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4894.479675] LNetError: 151145:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4894.506527] LNet: Removed LNI 192.168.202.110@tcp [ 4895.436592] Key type .llcrypt unregistered [ 4895.438647] Key type ._llcrypt unregistered [ 4924.186533] Key type ._llcrypt registered [ 4924.188824] Key type .llcrypt registered [ 4924.321992] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 4925.396592] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4925.415165] alg: No test for adler32 (adler32-zlib) [ 4926.551912] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 4926.856129] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 4928.561114] Key type lgssc registered [ 4929.853411] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4938.382804] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4938.398057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4939.812197] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4945.256599] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4949.986929] LustreError: 152560:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4950.019633] LustreError: 152560:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4955.113074] LustreError: 152561:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4955.179063] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4955.691071] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4959.879131] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 4968.822803] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4977.389721] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4986.326575] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4986.742464] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4993.296348] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5000.041562] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5001.212610] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5010.321347] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5034.212744] Lustre: lustre-MDT0001: evicting ec6e6dcd-1b1a-4387-a313-d5f08689448c (at 192.168.202.10@tcp) by administrative request [ 5034.220258] Lustre: Skipped 1 previous similar message [ 5066.723444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5066.732163] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5066.738776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5066.744200] Lustre: Skipped 1 previous similar message [ 5066.757874] LustreError: Skipped 1 previous similar message [ 5069.235805] Lustre: server umount lustre-OST0000 complete [ 5076.968343] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5076.972957] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5076.981961] Lustre: Skipped 1 previous similar message [ 5076.998802] Lustre: Skipped 2 previous similar messages [ 5082.082341] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5086.182916] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5086.203805] Lustre: Skipped 2 previous similar messages [ 5088.223168] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5088.574086] Lustre: server umount lustre-MDT0000 complete [ 5091.298307] LustreError: 153582:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5091.332305] LustreError: 153582:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5092.863628] LustreError: 152540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787865883 with bad export cookie 16583182140843000421 [ 5092.872805] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5092.879890] LustreError: 152540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5093.285932] Lustre: server umount lustre-MDT0001 complete [ 5105.780747] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 5108.827824] Key type lgssc unregistered [ 5109.071212] LNet: 156663:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5109.095983] LNetError: 156663:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5110.121359] LNet: Removed LNI 192.168.202.110@tcp [ 5111.283093] Key type .llcrypt unregistered [ 5111.300655] Key type ._llcrypt unregistered [ 5190.765488] Key type ._llcrypt registered [ 5190.767688] Key type .llcrypt registered [ 5190.852963] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 5191.558763] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5191.626135] alg: No test for adler32 (adler32-zlib) [ 5193.434120] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 5193.909173] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 5195.935180] Key type lgssc registered [ 5198.309329] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5211.851607] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5211.907680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5213.543802] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5218.602962] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5223.909180] LustreError: 159155:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5223.953114] LustreError: 159155:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5229.027351] LustreError: 159154:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5229.859434] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5230.463454] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5235.427077] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5243.650284] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5249.800318] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5259.452179] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5260.387530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5267.455575] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5275.760334] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5277.445932] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5283.969172] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5294.564451] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5294.580821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5294.591165] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5294.602541] Lustre: Skipped 1 previous similar message [ 5294.612990] Lustre: Skipped 1 previous similar message [ 5299.120818] Lustre: server umount lustre-OST0000 complete [ 5304.801832] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5304.803192] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5304.822551] Lustre: Skipped 1 previous similar message [ 5304.847161] Lustre: Skipped 1 previous similar message [ 5309.933604] LustreError: 159155:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5309.962134] LustreError: 159155:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5310.283651] Lustre: server umount lustre-MDT0000 complete [ 5314.019051] LustreError: 159149:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5314.040165] LustreError: 159149:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5315.404361] LustreError: 161009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866106 with bad export cookie 12733558132556827469 [ 5315.408656] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5315.413801] LustreError: 161009:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5316.250111] Lustre: server umount lustre-MDT0001 complete [ 5329.973619] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 5333.842781] Key type lgssc unregistered [ 5334.222549] LNet: 162725:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5334.251426] LNetError: 162725:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5335.320212] LNet: Removed LNI 192.168.202.110@tcp [ 5336.287193] Key type .llcrypt unregistered [ 5336.288966] Key type ._llcrypt unregistered [ 5362.511967] Key type ._llcrypt registered [ 5362.514899] Key type .llcrypt registered [ 5362.629319] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 5364.020199] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5364.157550] alg: No test for adler32 (adler32-zlib) [ 5365.321168] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 5365.635440] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 5367.319331] Key type lgssc registered [ 5368.490665] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5386.275685] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 17:29:36 (1787866176) [ 5403.774370] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 5415.934628] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 5462.501590] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5462.514464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5463.832965] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5463.858940] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5463.945639] Lustre: lustre-MDT0000: new disk, initializing [ 5464.021904] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5464.078135] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5469.029882] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5480.649700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5480.852838] Lustre: 166991:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5480.946247] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5480.951429] Lustre: Skipped 1 previous similar message [ 5481.062870] Lustre: lustre-MDT0001: new disk, initializing [ 5481.193498] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5481.274818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5481.291412] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5486.395195] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5495.946670] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5502.360672] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5509.371751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5509.623238] Lustre: lustre-OST0000: new disk, initializing [ 5509.627498] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5509.632080] Lustre: 168961:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5509.724269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5516.397044] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5516.402790] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5516.427792] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5516.579358] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5525.452108] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5537.249856] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5537.261470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5537.290393] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5541.526628] Lustre: server umount lustre-OST0000 complete [ 5547.490542] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5547.501102] Lustre: Skipped 1 previous similar message [ 5547.506765] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5547.515122] Lustre: Skipped 2 previous similar messages [ 5552.611498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5552.616332] Lustre: Skipped 1 previous similar message [ 5557.730495] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5557.747328] Lustre: Skipped 1 previous similar message [ 5560.288614] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5560.660746] Lustre: server umount lustre-MDT0000 complete [ 5564.640358] LustreError: 166984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787866355 with bad export cookie 3468199204943872299 [ 5564.649898] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5564.652761] LustreError: 166984:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5565.058351] Lustre: server umount lustre-MDT0001 complete [ 5577.411580] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 5580.909313] Key type lgssc unregistered [ 5581.131967] LNet: 170871:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5581.136950] LNetError: 170871:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5581.160185] LNet: Removed LNI 192.168.202.110@tcp [ 5582.395063] Key type .llcrypt unregistered [ 5582.396831] Key type ._llcrypt unregistered [ 5606.844391] Key type ._llcrypt registered [ 5606.847946] Key type .llcrypt registered [ 5606.982876] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 5607.949491] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5607.987379] alg: No test for adler32 (adler32-zlib) [ 5609.162685] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 5609.446356] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 5611.111777] Key type lgssc registered [ 5612.142579] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5619.893397] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5619.930819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5621.450851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5626.542398] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5631.974333] LustreError: 172285:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5631.991766] LustreError: 172285:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5636.092074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5636.513788] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5642.773464] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5651.113561] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5657.871448] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5664.802631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5665.155706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5667.208236] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5671.168262] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 5678.612025] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5687.780759] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5687.818039] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5687.843675] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5687.863823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.880889] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5688.745554] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5688.767943] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5688.793857] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5688.804485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.825618] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5689.740137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.568550] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5690.573051] Lustre: Skipped 1 previous similar message [ 5690.596326] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5690.603532] Lustre: Skipped 1 previous similar message [ 5690.606475] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5690.610207] Lustre: Skipped 1 previous similar message [ 5690.613114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.622446] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5690.628502] Lustre: Skipped 1 previous similar message [ 5691.394731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.183834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.979723] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5692.984387] Lustre: Skipped 2 previous similar messages [ 5692.995804] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5693.001738] Lustre: Skipped 2 previous similar messages [ 5693.004213] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5693.006623] Lustre: Skipped 2 previous similar messages [ 5693.008880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.018395] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5693.029237] Lustre: Skipped 2 previous similar messages [ 5693.860469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.736915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.594236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.698258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.667549] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5697.670838] Lustre: Skipped 4 previous similar messages [ 5697.682494] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5697.689595] Lustre: Skipped 4 previous similar messages [ 5697.693115] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5697.697181] Lustre: Skipped 4 previous similar messages [ 5697.703280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.718520] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5697.722642] Lustre: Skipped 4 previous similar messages [ 5698.710456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.554911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.406753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.214688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.240293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.175783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.182610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.110527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.971972] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5705.979569] Lustre: Skipped 8 previous similar messages [ 5706.002974] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5706.014566] Lustre: Skipped 8 previous similar messages [ 5706.019453] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5706.028303] Lustre: Skipped 8 previous similar messages [ 5706.031787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.041293] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5706.050064] Lustre: Skipped 8 previous similar messages [ 5706.934453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.883352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.763769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.784711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.623695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.605092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.361456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.191680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.107369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.019308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.956468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.862335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.847278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.668715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.476509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.366412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.381248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.514303] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5722.521181] Lustre: Skipped 17 previous similar messages [ 5722.539427] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5722.547717] Lustre: Skipped 17 previous similar messages [ 5722.550606] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5722.554412] Lustre: Skipped 17 previous similar messages [ 5722.557397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.565725] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5722.568265] Lustre: Skipped 17 previous similar messages [ 5723.383896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.268375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.165758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.216544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.186237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.255789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.232171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.163727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.053459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.005517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.793555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.577427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.452329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.356549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.180843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.040753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.808695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.727144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.606668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.376778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.139890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.894760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.670293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.647190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.676417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.698888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.922876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.718139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.517912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.258150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.220415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.210533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.031898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.865471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.708174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.480631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.280224] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5755.285939] Lustre: Skipped 36 previous similar messages [ 5755.294212] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5755.300753] Lustre: Skipped 36 previous similar messages [ 5755.302911] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5755.305836] Lustre: Skipped 36 previous similar messages [ 5755.308660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.319125] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5755.326605] Lustre: Skipped 36 previous similar messages [ 5755.935670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.653720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.421329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.176282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.007151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.978619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.917280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.853271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.714108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.608334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.508037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.393476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.524550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.638165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.634662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.637290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.525961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.356857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.215863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.120779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.369841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.315479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.291640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.338927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.550340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.598299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.364656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.174943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.073755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.343349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.280747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.352413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.144550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.966843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.686555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.484457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.317632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.249785] LustreError: 172305:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5790.257152] LustreError: 172305:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5790.270137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.126417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.040743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.909859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.996109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.998070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.937306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.874739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.792333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.680525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.600977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.540785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.735200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.900238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.036131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.069816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.262079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.593357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.549586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.974223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.668876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.683935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.631932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.560833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.344397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.272696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.296095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.152624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.049446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.841844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.534917] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5819.539069] Lustre: Skipped 67 previous similar messages [ 5819.552659] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5819.560237] Lustre: Skipped 67 previous similar messages [ 5819.564254] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5819.567419] Lustre: Skipped 67 previous similar messages [ 5819.572428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.582862] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5819.585831] Lustre: Skipped 67 previous similar messages [ 5820.467752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.318726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.382946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.364880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.285856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.273817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.432865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.382928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.327359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.285355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.427250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.555436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.625277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.819493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.870370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.825835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.764428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.752793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.775140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.740129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.595667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.609738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.508257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.495446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.713844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.634820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.804237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.643882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.511952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.515821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.390724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.412906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.296040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.317576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.266869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.278366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.192647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.114459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.107049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.115390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.224890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.231360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.361638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.225771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.426656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.415370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.717218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.484395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.485581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.414121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.247828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.106557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.061822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.301264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.353947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.403287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.572819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.510392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.769623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.934151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.064124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.277168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.317429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.282459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.462183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.241749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.303758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.325929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.284344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.296435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.212239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.061779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.061670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.991299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.899250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.078777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.089524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.361886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.718802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.513296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.742844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.559142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.470508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.287410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.086566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.051576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.416805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.351402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.169471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.022922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.290781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.356672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.385468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.848652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.074200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.316961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.782975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.121872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.301443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.150532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.948664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.766515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.598168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.529764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.661600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.718444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.637548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.500244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.441550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.233798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.232779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.158247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.083502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.804174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.836123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.672888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.489450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.249097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.227644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.008486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.082466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.892762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.169077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.173353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.136804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.090364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.270817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.335235] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5948.338282] Lustre: Skipped 127 previous similar messages [ 5948.354254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5948.365953] Lustre: Skipped 127 previous similar messages [ 5948.370475] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5948.380627] Lustre: Skipped 127 previous similar messages [ 5948.385158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.396645] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5948.413805] Lustre: Skipped 127 previous similar messages [ 5949.261006] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.288285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.896658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.851741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.786177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.606291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.537668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.389064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.340766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.143658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.887164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.775176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.682493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.659869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.837874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.752556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.619650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.393102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.321166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.276576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.250776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.058868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.805389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.724469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.706456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.843941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.857318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.883686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.931932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.021397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.013670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.135768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.294592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.401665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.549885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.565947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.758340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.520059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.398285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.037739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.838372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.599867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.304297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.141063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.886639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.622907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.427948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.264385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.104770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.954520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.845200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.663465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.608927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.324716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.039940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.116443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.340369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.085748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.250634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.241494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.477490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.351713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.466423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.247526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.533486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.892578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.121921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.104755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.328768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.332762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.222861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.024565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.990638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.847199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.688988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.499758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.290548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.058886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.810563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.616587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.690721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.564897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.574248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.509810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.634230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.578880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.467919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.357376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.292294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.004909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.925331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.846917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.867871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.845181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.829160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.584754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.619091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.686488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.640953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.869993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.911699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.005837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.943748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.115221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.007991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.046488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.065392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.070610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.298062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.473847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.604597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.210246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.115962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.929305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.972668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.134165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.087324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.233504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.953295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.838938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.539554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.383996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.438495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.594585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.037688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.441996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.954356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.288528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.553700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.396232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.417427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.541506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.576377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.585463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.687764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.795822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.919312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.060922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.010962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.009193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.008214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.218128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.345038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.596719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.929470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.094902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.096807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.162261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.302560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.286978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.406038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.300421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.498071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.396558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.229338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.179879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.639783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.575792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.398325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.208622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.502969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.739191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.614730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.809308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.581950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.332453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.163021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.152631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.937137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.613704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.492957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.799579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.625095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.590307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.633468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.489949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.599188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.599142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.079574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.901636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.186159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.737019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.244324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.999929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.557577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.656829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.614299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.738046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.733870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.961577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.987661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.432619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.819627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.048775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.176492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.205390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.236742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.446287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.431223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.700363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.558917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.643444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.564909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.796604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.871271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.822118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.005889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.957966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.109217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.110762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.834918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.731812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.727365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.885509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.117770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.047854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.512233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.424716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.607781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.628292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.765490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.788870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.783230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.617288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.474618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.374395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.348759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.234954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.237392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.107936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.916366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.751078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.633768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.612177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.447758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.565538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.710099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.745953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.673287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.383892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.092408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.989490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.778713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.650596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.630564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.423861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.352534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.137945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.965578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.919238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.691760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.502748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.720238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.703840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.692804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.548791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.386335] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6204.391751] Lustre: Skipped 256 previous similar messages [ 6204.403050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6204.411111] Lustre: Skipped 256 previous similar messages [ 6204.415143] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6204.420183] Lustre: Skipped 256 previous similar messages [ 6204.424505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.436032] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6204.444190] Lustre: Skipped 256 previous similar messages [ 6205.316659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.461061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.543474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.803704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.678191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.701874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.591395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.739762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.711958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.723375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.571596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.659924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.638909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.775280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.804578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.623829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.543063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.501305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.404495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.342477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.154922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.258181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.051942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.119572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.031806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.950761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.883819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.953181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.037756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.889812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.652760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.607594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.470881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.488209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.361742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.351967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.435458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.568243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.750838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.633070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.656107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.715683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.703949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.038757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.825574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.095533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.710041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.780207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.863396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.820933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.700668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.868456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.732481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.709402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.850214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.773956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.865763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.785276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.808377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.670804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.685113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.877848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.785749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.825546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.862399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.724564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.738457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.540142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.465490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.432375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.215135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.287484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.261043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.129692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.907699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.886580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.807675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.721318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.559152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.443351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.293145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.364211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.373671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.468094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.643192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.861957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.841341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.781777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.874785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.757460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.648286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.669129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.665579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.574458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.542962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.553744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.730125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.516358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.633880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.689645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.059469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.578157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.878083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.019994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.841876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.564464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.487896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.921636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.926753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.033086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.168588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.485552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.688781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.757215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.754152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.972321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.037176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.998116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.076817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.240617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.494725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.647833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.759730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.756748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.767720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.731263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.007808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.168266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.299292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.321542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.372712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.676099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.901911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.939827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.943086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.968245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.133444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.088169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.072336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.988623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.237846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.347042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.520924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.534448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.485279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.594295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.515827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.587702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.614242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.620806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.761594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.782584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.038730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.223421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.334495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.241412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.272623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.216576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.166632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.103751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.160593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.136357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.148403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.922376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.043619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.917716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.968326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.980730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.041861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.877186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.762519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.856643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.029509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.203404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.119748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.324178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.428100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.599969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.850855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.904271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.964634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.878992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.646739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.531514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.316220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.057608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.802404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.472958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.337069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.145658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.888438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.720912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.684975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.718347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.619549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.673351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.914285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.922514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.929626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.060976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.258850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.344624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.304141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.514554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.687389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.962782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.239932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.521286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.621869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.739895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.886285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.871674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.986123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.990080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.026862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.908231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.077681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.052158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.147213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.167753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.094409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.988838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.036902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.753510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.780302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.864467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.728277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.655762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.435590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.380798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.275596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.293570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.299910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.366755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.431769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.219849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.052592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.035643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.810603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.436314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.138726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.962513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.642876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.503723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.749663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.101955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.226199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.447844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.212218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.713408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.348565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.638304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.858659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.185083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.562761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.834026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.785785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.150973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.125321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.917280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.886355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.289749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.325629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.202233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.297757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.426509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.371911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.540846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.529971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.506480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.423475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.497566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.679233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.656830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.796281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.790991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.844200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.940957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.150887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.298449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.581583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.958138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.047809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.076153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.075451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.784407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.675043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.520610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.604833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.218399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.283285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.342252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.300230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.246255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.266538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.330553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.734248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.266093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.478598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.802814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.499373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.631097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.499209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.535647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.722325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.761646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.907313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.736138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.506408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.197543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.901759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.593337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.577362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.656936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.621235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.871543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.121295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.169293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.250036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.476257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.715827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.959802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.378055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.371264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.551179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.367164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.059824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.846756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.737727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.582977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.410149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.283535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.504388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.359413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.259248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.038674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.110559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.106776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.010501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.821871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.533587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.427556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.470589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.373182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.167133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.047703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.973976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.776209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.730790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.839463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.836727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.887335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.833739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.924517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.893702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.900699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.052758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.171450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.207777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.366812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.464750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.466643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.675226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.619300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.577462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.446844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.551570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.453565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.417148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.392597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.371596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.375815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.698635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.576723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.816394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.245860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.600209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.746350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.143930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.248643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.489413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.840678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.192984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.434668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.843726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.275250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.519665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.861655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.829589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.888043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.325038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.719765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.853553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.212286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.072752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.976330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.015949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.198583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.286556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.465845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.582113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.750703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.637629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.768243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.847097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.860986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.064750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.035069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.021941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.081102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.114525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.275217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.389686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.583777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.487642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.199053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.493054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.398760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.274076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.284489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.102296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.204798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.229221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.504298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.966532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.920967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.133633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.410377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.776215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.130323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.215872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.366056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.501842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.237325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.667069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.614940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.551739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.675691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.536960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.709586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.728417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.954139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.912743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.934439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.658132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.594403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.367358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.079716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.017804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.730342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.520816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.724550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.178723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.692463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.872181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.371844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.548674] LustreError: 172305:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 6681.568089] LustreError: 172305:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 6681.581449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.760201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.899943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.810445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.836929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.012485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.930151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.995634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.794895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.178561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.350863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.686434] LustreError: 172305:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6693.700746] LustreError: 172305:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6693.703741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.876565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.964795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.070344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7180.149533] LustreError: 174108:0:(ofd_dev.c:1794:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7180.153993] LustreError: 172305:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7886.830977] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7886.834365] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7886.845977] LustreError: Skipped 1 previous similar message [ 7886.861909] Lustre: Skipped 471 previous similar messages [ 7886.871115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7888.137953] Lustre: server umount lustre-OST0000 complete [ 7894.496288] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7894.508723] LustreError: Skipped 6 previous similar messages [ 7894.537110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7894.539958] Lustre: Skipped 2 previous similar messages [ 7897.570613] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7897.578160] Lustre: Skipped 1 previous similar message [ 7898.962404] Lustre: server umount lustre-MDT0000 complete [ 7902.617989] LustreError: 172266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868693 with bad export cookie 8468619577752863189 [ 7902.626135] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7902.626929] LustreError: 172266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7903.071654] Lustre: server umount lustre-MDT0001 complete [ 7917.493934] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 7920.976578] Key type lgssc unregistered [ 7921.467052] LNet: 227581:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7921.476511] LNetError: 227581:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7921.498986] LNet: Removed LNI 192.168.202.110@tcp [ 7922.828986] Key type .llcrypt unregistered [ 7922.830958] Key type ._llcrypt unregistered [ 7954.933697] Key type ._llcrypt registered [ 7954.940787] Key type .llcrypt registered [ 7955.083428] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 7956.812091] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7956.908923] alg: No test for adler32 (adler32-zlib) [ 7958.036372] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 7958.392194] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 7960.193121] Key type lgssc registered [ 7961.910773] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7972.536778] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7972.567210] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7974.128952] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7980.413850] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 7984.610025] LustreError: 229000:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7984.644731] LustreError: 229000:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 7989.737727] LustreError: 229001:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7990.624532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7991.087247] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7997.353825] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8006.850914] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8014.697876] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8023.366441] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8023.999328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8031.899071] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8041.146236] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8044.598302] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49664 to 0x280000401:49695) [ 8050.658923] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8052.515426] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8052.532330] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8052.539066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8052.562033] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8052.562634] LustreError: 229021:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8054.763375] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8054.771739] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8054.784104] Lustre: Skipped 1 previous similar message [ 8054.802712] Lustre: Skipped 1 previous similar message [ 8057.163474] Lustre: lustre-OST0000: Not available for connect from 192.168.202.10@tcp (stopping) [ 8059.888148] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8059.898017] Lustre: Skipped 1 previous similar message [ 8060.921302] Lustre: server umount lustre-OST0000 complete [ 8072.647253] LustreError: 229021:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8097.245421] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8097.834059] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8104.924851] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8110.123414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8110.136863] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8110.146745] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8110.174466] Lustre: Skipped 1 previous similar message [ 8113.621211] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8124.238510] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 8 sec [ 8130.545379] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff968303b8b800.ost_server_uuid 50 [ 8132.182724] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff968303b8b800.ost_server_uuid in FULL state after 0 sec [ 8135.649853] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8135.653696] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8135.658832] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8140.774131] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8140.776569] Lustre: Skipped 1 previous similar message [ 8142.168513] Lustre: server umount lustre-OST0000 complete [ 8149.993196] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8150.002839] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8150.028590] Lustre: Skipped 1 previous similar message [ 8150.044828] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8161.247685] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8161.715820] Lustre: server umount lustre-MDT0000 complete [ 8165.348417] LustreError: 229001:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8165.372585] LustreError: 229001:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8166.982339] LustreError: 228982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787868958 with bad export cookie 15943078905532840279 [ 8166.982905] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8166.992988] LustreError: 228982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8167.428649] Lustre: server umount lustre-MDT0001 complete [ 8178.940340] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 8181.366158] Key type lgssc unregistered [ 8181.732028] LNet: 234220:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8181.738913] LNetError: 234220:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8181.752918] LNet: Removed LNI 192.168.202.110@tcp [ 8182.884427] Key type .llcrypt unregistered [ 8182.887411] Key type ._llcrypt unregistered [ 8218.132359] Key type ._llcrypt registered [ 8218.134526] Key type .llcrypt registered [ 8218.306107] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 8220.097455] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8220.184334] alg: No test for adler32 (adler32-zlib) [ 8221.324442] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 8221.609857] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 8223.279195] Key type lgssc registered [ 8224.746811] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8236.079554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8236.092249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8237.711476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8243.037627] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8247.284679] LustreError: 235952:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8247.310646] LustreError: 235952:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8252.388752] LustreError: 235953:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8253.631423] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8254.067790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8259.350993] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8262.333950] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8270.576538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8271.241118] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8279.631265] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8282.510571] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49727) [ 8290.328699] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8290.649957] Lustre: lustre-OST0001: new disk, initializing [ 8290.660781] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8290.677695] Lustre: 238270:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8290.942321] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8296.603621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8296.622943] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8296.791536] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8299.852758] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8312.920332] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8321.020741] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8337.888424] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8337.901952] Lustre: Skipped 3 previous similar messages [ 8337.903381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8337.913134] Lustre: Skipped 3 previous similar messages [ 8343.018081] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8343.025426] Lustre: Skipped 3 previous similar messages [ 8343.893151] Lustre: server umount lustre-MDT0000 complete [ 8348.134058] LustreError: 235953:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8348.166888] LustreError: 235953:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 8348.284723] LustreError: 235933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869139 with bad export cookie 15130435618502820722 [ 8348.302086] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8348.303468] LustreError: 235933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8348.649641] Lustre: server umount lustre-MDT0001 complete [ 8363.774294] Lustre: server umount lustre-OST0000 complete [ 8378.234705] Lustre: server umount lustre-OST0001 complete [ 8407.584915] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 8420.477948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8421.082188] LustreError: 241378:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8421.188141] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8426.468500] LustreError: 241379:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8426.538471] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8436.119502] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8436.720184] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8442.396652] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8445.564627] Lustre: 242520:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8453.518253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8453.861880] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8461.040294] LustreError: 242874:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8461.066608] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8461.073768] LustreError: 242874:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8461.470707] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8466.189822] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49759) [ 8471.806288] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8479.594175] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8487.822902] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8492.487981] Lustre: 244393:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8507.878304] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8507.879686] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8507.890404] Lustre: Skipped 3 previous similar messages [ 8507.904353] Lustre: Skipped 3 previous similar messages [ 8512.659191] Lustre: server umount mylustre-MDT0000 complete [ 8513.000617] LustreError: 242896:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8513.016829] LustreError: 242896:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [ 8517.010610] LustreError: 241354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869308 with bad export cookie 15130435618502823585 [ 8517.018796] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8517.026499] LustreError: 241354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8517.860466] Lustre: server umount mylustre-MDT0001 complete [ 8533.523108] Lustre: server umount mylustre-OST0000 complete [ 8534.367561] Lustre: 235215:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787869309/real 1787869309] req@ffff9ff1092a4000 x1874716537295616/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787869325 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8534.408252] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8538.332423] Lustre: server umount mylustre-OST0001 complete [ 8565.056328] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 8578.000753] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8578.313232] LustreError: 246720:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8578.380961] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8578.384996] Lustre: Skipped 1 previous similar message [ 8584.650569] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8595.045395] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8601.348264] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8605.219269] Lustre: 247868:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8612.901305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8620.679338] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8621.444878] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8623.748901] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49791) [ 8632.386390] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8640.494249] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8649.129621] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8653.768463] Lustre: 249739:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8661.985628] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8661.996903] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8662.020179] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8668.528314] Lustre: server umount tfs-MDT0000 complete [ 8668.646280] LustreError: 246721:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8668.687534] LustreError: 246721:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 11 previous similar messages [ 8673.412291] LustreError: 246696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869464 with bad export cookie 15130435618502825538 [ 8673.412967] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8673.437776] LustreError: 246696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8673.762825] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8673.770133] Lustre: Skipped 3 previous similar messages [ 8673.863132] Lustre: server umount tfs-MDT0001 complete [ 8680.271145] Lustre: server umount tfs-OST0000 complete [ 8684.653233] Lustre: server umount tfs-OST0001 complete [ 8708.873781] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 8720.727454] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8721.343536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8721.355484] Lustre: Skipped 3 previous similar messages [ 8725.837427] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8736.492643] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8742.196795] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8745.557340] Lustre: 253130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8752.606707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8760.487457] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8761.279635] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8766.418084] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49823) [ 8770.623914] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8779.170096] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8788.106049] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8792.552124] Lustre: 255005:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8801.185184] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 18:26:32 (1787869592) [ 8818.662390] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8818.667135] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8818.680425] Lustre: Skipped 1 previous similar message [ 8818.686739] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8818.693588] Lustre: Skipped 4 previous similar messages [ 8823.939576] Lustre: server umount lustre-MDT0000 complete [ 8825.316526] LustreError: 251990:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8825.337361] LustreError: 251990:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 12 previous similar messages [ 8828.032525] LustreError: 253132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869619 with bad export cookie 15130435618502827232 [ 8828.036222] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8828.044787] LustreError: 253132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8830.439247] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8830.454161] Lustre: Skipped 5 previous similar messages [ 8834.483417] Lustre: server umount lustre-MDT0001 complete [ 8844.792083] Lustre: server umount lustre-OST0000 complete [ 8855.512080] Lustre: server umount lustre-OST0001 complete [ 8863.880630] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 8875.844639] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 8920.357382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8920.709374] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8920.758196] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8920.904907] Lustre: lustre-MDT0000: new disk, initializing [ 8921.063585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8921.076684] Lustre: Skipped 3 previous similar messages [ 8921.099190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8925.503758] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8939.589472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8939.926489] Lustre: 258189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8940.073587] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8940.079565] Lustre: Skipped 1 previous similar message [ 8940.214470] Lustre: lustre-MDT0001: new disk, initializing [ 8940.445186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8940.459507] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8946.582316] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8958.603862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8958.956731] Lustre: lustre-OST0000: new disk, initializing [ 8958.964270] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8958.977939] Lustre: 259771:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8960.535096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8960.554581] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8960.685328] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8967.201748] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 8979.043184] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8979.198367] Lustre: lustre-OST0001: new disk, initializing [ 8979.201111] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8979.205277] Lustre: 260644:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8981.158067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8981.171316] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8981.265902] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8986.490810] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9000.440115] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 18:29:50 (1787869790) [ 9017.081819] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 18:30:07 (1787869807) [ 9032.161725] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9032.164139] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9032.172881] Lustre: Skipped 5 previous similar messages [ 9032.178464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9032.184071] Lustre: Skipped 2 previous similar messages [ 9038.119680] Lustre: server umount lustre-OST0000 complete [ 9042.920521] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9042.929531] LustreError: Skipped 1 previous similar message [ 9048.764303] Lustre: server umount lustre-MDT0000 complete [ 9052.843293] LustreError: 258181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787869843 with bad export cookie 15130435618502828786 [ 9052.845777] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9052.855387] LustreError: 258181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9059.220934] Lustre: server umount lustre-MDT0001 complete [ 9070.487979] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 9143.832112] Lustre: server umount lustre-OST0001 complete [ 9151.131850] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 9165.763969] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 9204.382554] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9204.641717] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9204.681751] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9204.758349] Lustre: lustre-MDT0000: new disk, initializing [ 9204.846562] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9204.851801] Lustre: Skipped 3 previous similar messages [ 9204.873923] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9209.647578] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9221.844300] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9221.963219] Lustre: 265492:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9222.007319] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9222.009840] Lustre: Skipped 1 previous similar message [ 9222.074475] Lustre: lustre-MDT0001: new disk, initializing [ 9222.200867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9222.208314] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9226.612400] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9236.610966] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9244.828683] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9254.645427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9254.917071] Lustre: lustre-OST0000: new disk, initializing [ 9254.920413] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9254.927804] Lustre: 267461:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9256.091743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9256.098649] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9256.215440] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9262.604506] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9272.192576] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9282.016326] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9282.025823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9282.029502] LustreError: Skipped 1 previous similar message [ 9282.030810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9282.030817] Lustre: Skipped 11 previous similar messages [ 9282.070802] Lustre: Skipped 6 previous similar messages [ 9286.433207] Lustre: server umount lustre-OST0000 complete [ 9296.769811] Lustre: server umount lustre-MDT0000 complete [ 9297.378933] LustreError: 265497:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9297.400761] LustreError: 265497:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 9 previous similar messages [ 9301.480978] LustreError: 266286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870092 with bad export cookie 15130435618502830256 [ 9301.485286] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9301.497576] LustreError: 266286:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9301.848204] Lustre: server umount lustre-MDT0001 complete [ 9314.819745] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 9319.060978] Key type lgssc unregistered [ 9319.473292] LNet: 269223:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9319.486675] LNetError: 269223:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9320.570419] LNet: Removed LNI 192.168.202.110@tcp [ 9321.566155] Key type .llcrypt unregistered [ 9321.568187] Key type ._llcrypt unregistered [ 9349.443285] Key type ._llcrypt registered [ 9349.444442] Key type .llcrypt registered [ 9349.601887] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 9350.877134] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9351.192077] alg: No test for adler32 (adler32-zlib) [ 9352.732911] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 9352.987804] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 9354.703189] Key type lgssc registered [ 9356.357628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9365.734797] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9365.760155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9367.405494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9372.420483] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9377.768641] LustreError: 270690:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9377.780804] LustreError: 270690:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 9382.885531] LustreError: 270691:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9382.921558] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9383.400092] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9389.029032] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9398.104519] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9405.034787] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9413.750966] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9414.229253] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9421.728824] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9429.874816] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9431.213276] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9434.082243] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9434.104870] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9435.435802] Lustre: server umount lustre-OST0000 complete [ 9439.720055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9439.731020] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9439.760781] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9442.273191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9442.275131] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9445.917996] Lustre: server umount lustre-MDT0000 complete [ 9447.399693] LustreError: 270691:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9447.409751] LustreError: 270691:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 9450.565287] LustreError: 270671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870241 with bad export cookie 7383198822635319478 [ 9450.570868] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9450.575080] LustreError: 270671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9450.984085] Lustre: server umount lustre-MDT0001 complete [ 9465.755432] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 9469.140526] Key type lgssc unregistered [ 9469.460523] LNet: 274277:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9469.482513] LNetError: 274277:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9469.515211] LNet: Removed LNI 192.168.202.110@tcp [ 9470.846103] Key type .llcrypt unregistered [ 9470.856926] Key type ._llcrypt unregistered [ 9489.687846] Key type ._llcrypt registered [ 9489.690319] Key type .llcrypt registered [ 9489.807396] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 9491.071833] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9491.108263] alg: No test for adler32 (adler32-zlib) [ 9492.347347] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 9492.658415] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 9494.425090] Key type lgssc registered [ 9495.966794] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9505.786848] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9505.803203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9521.311474] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9526.432239] LustreError: 275473:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.110@tcp: failed processing log, type 4: rc = -110 [ 9553.057361] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9559.704012] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9568.502703] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9570.901564] Lustre: server umount lustre-OST0000 complete [ 9588.925951] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [ 9592.759512] Key type lgssc unregistered [ 9593.125631] LNet: 277106:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9593.133536] LNetError: 277106:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9593.149291] LNet: Removed LNI 192.168.202.110@tcp [ 9594.227761] Key type .llcrypt unregistered [ 9594.230180] Key type ._llcrypt unregistered [ 9637.394491] Key type ._llcrypt registered [ 9637.398266] Key type .llcrypt registered [ 9637.601934] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 9639.173696] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9639.304846] alg: No test for adler32 (adler32-zlib) [ 9640.856653] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [ 9641.298200] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [ 9643.127149] Key type lgssc registered [ 9644.393416] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9657.085456] loop0: detected capacity change from 0 to 204800000 [ 9657.227842] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9657.236205] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9657.493546] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9658.481458] loop0: detected capacity change from 0 to 204800000 [ 9658.690955] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9658.716420] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9659.119181] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9660.302572] loop0: detected capacity change from 0 to 204800000 [ 9660.449214] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9660.469985] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9660.577513] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9661.725395] loop0: detected capacity change from 0 to 204800000 [ 9661.979526] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9663.567304] loop0: detected capacity change from 0 to 204800000 [ 9663.612238] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9668.304813] loop0: detected capacity change from 0 to 204800000 [ 9668.335880] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9672.959386] loop0: detected capacity change from 0 to 204800000 [ 9673.004052] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9677.795824] loop0: detected capacity change from 0 to 204800000 [ 9677.829212] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9682.429899] loop0: detected capacity change from 0 to 204800000 [ 9682.529651] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9682.604517] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9682.680547] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9682.689750] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9682.701687] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9682.710412] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9682.719152] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9682.729598] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9682.737931] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9682.746119] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9682.760943] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9682.773675] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9682.783845] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9682.794505] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9682.807696] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9682.820642] Lustre: 280543:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9689.481187] Lustre: server umount MGS complete [ 9690.677378] loop0: detected capacity change from 0 to 204800000 [ 9690.794086] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9691.360519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9691.415484] Lustre: lustre-MDT0000: Aborting client recovery [ 9692.516350] loop1: detected capacity change from 0 to 204800000 [ 9692.671322] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9692.779535] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9692.786945] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9692.793733] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9692.802206] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9692.808307] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9692.816502] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9692.822650] Lustre: 280976:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9692.831276] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9692.947609] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9692.990927] Lustre: lustre-MDT0001: Aborting client recovery [ 9693.982716] loop2: detected capacity change from 0 to 204800000 [ 9694.137873] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9694.175427] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9694.197591] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9694.214634] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9694.231292] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9694.243232] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9694.257507] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9694.267963] Lustre: 281055:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9694.366146] Lustre: Found index 0 for lustre-OST0000, updating log [ 9694.414372] Lustre: 281074:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9694.553376] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9694.579380] Lustre: lustre-OST0000: Aborting recovery [ 9695.594183] loop3: detected capacity change from 0 to 204800000 [ 9695.698105] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9695.865665] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9695.884091] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9695.902338] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9695.918039] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9695.936649] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9695.948953] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9695.962601] Lustre: 281139:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9695.981441] Lustre: Found index 1 for lustre-OST0001, updating log [ 9696.014478] Lustre: 281144:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9696.082065] Lustre: lustre-OST0001: Aborting recovery [ 9701.248745] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9701.257415] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9701.292265] LustreError: 280920:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9701.329983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9701.344588] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9702.263279] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9702.264094] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9704.698579] LustreError: 280887:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1932us: rc = -2 [ 9705.238057] LustreError: 280884:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 3143us: rc = -2 [ 9705.269416] LustreError: 280884:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 13 previous similar messages [ 9718.755929] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9718.762193] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9718.781116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9722.848587] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9722.848627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9722.874720] Lustre: Skipped 2 previous similar messages [ 9722.877308] Lustre: Skipped 2 previous similar messages [ 9726.680441] Lustre: server umount lustre-MDT0000 complete [ 9727.976937] LustreError: 280890:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9728.014885] LustreError: 280890:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [ 9728.315148] LustreError: 280981:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870519 with bad export cookie 8677522665537769058 [ 9728.364083] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9733.089102] LustreError: 280886:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9733.099492] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9733.107835] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9733.107848] Lustre: Skipped 1 previous similar message [ 9733.121952] LustreError: 280886:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 9733.200173] Lustre: Skipped 1 previous similar message [ 9735.111802] Lustre: server umount lustre-MDT0001 complete [ 9742.989622] Lustre: server umount lustre-OST0000 complete [ 9750.388873] Lustre: server umount lustre-OST0001 complete [ 9759.279958] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 18:42:29 (1787870549) [ 9796.223592] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [ 9808.505415] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [ 9848.676353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9849.224639] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9849.301474] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9849.481602] Lustre: lustre-MDT0000: new disk, initializing [ 9849.597479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9849.603562] Lustre: Skipped 1 previous similar message [ 9849.636774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9855.697296] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9867.697900] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9867.848567] Lustre: 285209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9867.941316] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9867.946883] Lustre: Skipped 1 previous similar message [ 9868.078762] Lustre: lustre-MDT0001: new disk, initializing [ 9868.244757] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9868.278623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9868.291216] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9873.167572] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9883.095474] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9889.959146] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9896.043239] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9896.377842] Lustre: lustre-OST0000: new disk, initializing [ 9896.385317] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9896.395643] Lustre: 287179:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9896.507925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9904.118770] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [ 9904.663542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9904.671275] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9904.755566] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9915.771569] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9921.926785] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9934.736804] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9938.778564] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9946.696817] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9996.769656] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9996.772130] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9996.804542] Lustre: Skipped 1 previous similar message [ 9996.818761] Lustre: Skipped 1 previous similar message [ 9998.973345] Lustre: server umount lustre-OST0000 complete [10006.501940] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10006.515085] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10006.545581] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10006.555530] Lustre: Skipped 1 previous similar message [10008.788874] Lustre: server umount lustre-MDT0000 complete [10012.132185] LustreError: 285216:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10012.154222] LustreError: 285216:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10012.661460] LustreError: 287178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870803 with bad export cookie 8677522665537771767 [10012.669143] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10012.675293] LustreError: 287178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10013.087637] Lustre: server umount lustre-MDT0001 complete [10022.100094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10026.814080] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10032.024339] Lustre: server umount MGS complete [10042.792248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10043.196837] LustreError: 291000:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10043.264524] LustreError: 291013:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10043.317617] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10046.625397] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10048.481161] LustreError: 291001:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10054.292524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10058.772461] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10067.464985] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10073.384851] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10081.427974] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10081.804023] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10081.807886] Lustre: Skipped 1 previous similar message [10087.922272] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10096.481858] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10135.019158] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10135.021405] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10135.027946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10135.027956] Lustre: Skipped 2 previous similar messages [10135.029348] LustreError: Skipped 1 previous similar message [10135.067121] Lustre: Skipped 2 previous similar messages [10139.764839] Lustre: server umount lustre-OST0000 complete [10150.841917] Lustre: server umount lustre-MDT0000 complete [10154.794782] LustreError: 290982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787870945 with bad export cookie 8677522665537773622 [10154.795987] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10154.802247] LustreError: 290982:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10155.126531] Lustre: server umount lustre-MDT0001 complete [10166.714104] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [10169.975120] Key type lgssc unregistered [10170.363597] LNet: 294947:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10170.370366] LNetError: 294947:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10170.397372] LNet: Removed LNI 192.168.202.110@tcp [10171.562879] Key type .llcrypt unregistered [10171.566220] Key type ._llcrypt unregistered [10210.131542] Key type ._llcrypt registered [10210.134187] Key type .llcrypt registered [10210.282515] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [10225.733846] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [10226.691114] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10226.778409] alg: No test for adler32 (adler32-zlib) [10228.215142] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [10228.563241] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [10230.223504] Key type lgssc registered [10231.476611] Lustre: Echo OBD driver; http://www.lustre.org/ [10275.370661] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10275.427266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10276.750493] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10276.795431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10276.912517] Lustre: lustre-MDT0000: new disk, initializing [10277.034800] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10277.051638] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10282.906445] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10295.723291] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10295.882412] Lustre: 298910:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10295.934512] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10295.939294] Lustre: Skipped 1 previous similar message [10296.049714] Lustre: lustre-MDT0001: new disk, initializing [10296.189623] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10296.258909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10296.271294] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10301.820751] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10313.599855] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10321.263157] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10329.810884] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10330.171429] Lustre: lustre-OST0000: new disk, initializing [10330.175181] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10330.185136] Lustre: 300878:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10330.300723] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10337.578476] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10338.417090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10338.432644] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10338.608987] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10348.500935] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10355.431779] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10364.948919] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10372.122218] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10377.119350] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10425.824479] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10425.828934] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10425.838821] LustreError: Skipped 1 previous similar message [10425.839760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10425.870811] Lustre: Skipped 1 previous similar message [10429.615570] Lustre: server umount lustre-OST0000 complete [10434.530074] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10434.540752] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10434.565047] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10434.569410] Lustre: Skipped 1 previous similar message [10436.068637] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10436.072702] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10436.094560] Lustre: Skipped 1 previous similar message [10441.184088] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10441.194426] Lustre: Skipped 1 previous similar message [10445.295460] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10445.307580] Lustre: Skipped 1 previous similar message [10448.351348] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10448.592712] Lustre: server umount lustre-MDT0000 complete [10452.980826] LustreError: 300877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871244 with bad export cookie 10701823659677848871 [10452.994496] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10453.001178] LustreError: 300877:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10453.738215] Lustre: server umount lustre-MDT0001 complete [10464.268710] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10470.005798] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10475.908532] Lustre: server umount MGS complete [10489.707309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10490.269188] LustreError: 304697:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10490.301167] LustreError: 304697:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10490.309518] LustreError: 304710:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10490.408195] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10495.458749] LustreError: 304698:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10496.533391] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10500.584050] LustreError: 304697:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.702805] LustreError: 304698:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10507.475234] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10507.916563] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10512.860779] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10521.608961] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10527.019812] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10535.146328] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10535.369760] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10544.814576] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10555.082398] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10587.616359] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10587.622740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10587.650875] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10590.874382] Lustre: server umount lustre-OST0000 complete [10598.883953] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10598.892267] Lustre: Skipped 2 previous similar messages [10601.623248] Lustre: server umount lustre-MDT0000 complete [10604.006534] LustreError: 304698:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10604.032884] LustreError: 304698:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10606.436957] LustreError: 304678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787871397 with bad export cookie 10701823659677850712 [10606.445248] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10606.446663] LustreError: 304678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10606.885225] Lustre: server umount lustre-MDT0001 complete [10618.698213] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [10621.967626] Key type lgssc unregistered [10622.519273] LNet: 308650:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10622.525436] LNetError: 308650:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10623.589222] LNet: Removed LNI 192.168.202.110@tcp [10624.638948] Key type .llcrypt unregistered [10624.641126] Key type ._llcrypt unregistered [10647.096616] Key type ._llcrypt registered [10647.098490] Key type .llcrypt registered [10647.294341] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [10663.807480] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [10665.464607] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10665.501428] alg: No test for adler32 (adler32-zlib) [10667.056940] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [10667.411714] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [10669.167223] Key type lgssc registered [10670.916235] Lustre: Echo OBD driver; http://www.lustre.org/ [10700.645216] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [10719.094247] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10719.168841] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10720.599887] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10720.649855] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10720.736255] Lustre: lustre-MDT0000: new disk, initializing [10720.872131] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10720.908818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10726.392606] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10739.123078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10739.484182] Lustre: lustre-OST0000: new disk, initializing [10739.490515] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10739.494647] Lustre: Skipped 1 previous similar message [10739.499789] Lustre: 312703:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10739.582682] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10746.865893] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [10746.983837] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10746.991263] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10747.032144] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11714.528465] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11714.560413] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11715.250328] Lustre: server umount lustre-MDT0000 complete [11720.450941] LustreError: 312702:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872511 with bad export cookie 1945996856796175546 [11720.491402] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11720.955274] Lustre: server umount lustre-OST0000 complete [11754.764954] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [11767.975752] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [11813.329742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11813.590800] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11813.638644] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11813.739350] Lustre: lustre-MDT0000: new disk, initializing [11813.902775] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11813.923915] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11818.805078] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [11830.410716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11830.517833] Lustre: 317066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11830.566844] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11830.571625] Lustre: Skipped 1 previous similar message [11830.736828] Lustre: lustre-MDT0001: new disk, initializing [11830.752290] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11830.873371] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11830.912592] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11830.929827] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11836.367531] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [11848.001514] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11853.361766] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11861.006994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11861.462535] Lustre: lustre-OST0000: new disk, initializing [11861.465935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11861.475531] Lustre: 319038:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11861.577516] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11870.349118] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11870.361817] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11870.544906] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11870.556202] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [11882.434677] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11888.045528] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11888.313595] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11893.811985] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11894.119971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11901.409607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11901.413032] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11901.422344] LustreError: Skipped 1 previous similar message [11901.423299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11901.452096] Lustre: Skipped 1 previous similar message [11902.881417] Lustre: server umount lustre-OST0000 complete [11907.568299] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11907.580995] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11907.601216] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11907.608981] Lustre: Skipped 1 previous similar message [11911.648516] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11911.653649] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11921.375147] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11921.845472] Lustre: server umount lustre-MDT0000 complete [11926.949261] LustreError: 317863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872717 with bad export cookie 1945996856799116365 [11926.965042] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11926.968948] LustreError: 317863:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [11927.373932] Lustre: server umount lustre-MDT0001 complete [11932.221414] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11934.685548] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:18:44 (1787872724) [11942.473394] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11942.912767] LustreError: 321352:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11942.927112] LustreError: 321352:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [11942.991639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11948.002182] LustreError: 321353:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11948.647989] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [11951.008375] LustreError: 321352:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11957.222366] LustreError: 321353:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11957.233970] LustreError: 321353:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [11958.336443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11958.995758] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11964.142247] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [11971.828429] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11978.309658] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11985.965828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11986.539466] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11993.786777] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12002.246854] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12009.506588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12009.743407] Lustre: lustre-OST0001: new disk, initializing [12009.747191] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12009.754039] Lustre: 324191:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12009.913084] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12015.150208] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12015.155983] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12015.196529] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12016.229382] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12016.236897] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12017.163696] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12017.265314] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12017.282697] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12019.364281] LustreError: 324454:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12019.386757] LustreError: 324454:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 4 previous similar messages [12023.462284] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12023.492017] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 6 previous similar messages [12028.089629] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12028.360150] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12031.713257] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12031.723794] LustreError: 324448:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12034.930690] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9683056a2800.ost_server_uuid 50 [12036.958793] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9683056a2800.ost_server_uuid in FULL state after 0 sec [12044.146818] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9683056a2800.ost_server_uuid 50 [12045.892907] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9683056a2800.ost_server_uuid in FULL state after 0 sec [12048.240217] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12061.681611] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12061.710301] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12061.716910] Lustre: Skipped 4 previous similar messages [12067.209370] Lustre: server umount lustre-OST0001 complete [12071.911525] LustreError: 323174:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12071.943813] LustreError: 323174:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12074.976245] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12074.986455] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12074.997890] Lustre: Skipped 1 previous similar message [12079.457857] Lustre: server umount lustre-OST0000 complete [12086.751944] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12086.759383] LustreError: Skipped 1 previous similar message [12086.767394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12086.775289] Lustre: Skipped 8 previous similar messages [12090.332319] Lustre: server umount lustre-MDT0000 complete [12092.384951] LustreError: 321352:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12092.427430] LustreError: 321352:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12095.217310] LustreError: 321333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787872886 with bad export cookie 1945996856799117247 [12095.227731] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12095.235064] LustreError: 321333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12095.652112] Lustre: server umount lustre-MDT0001 complete [12108.631912] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [12111.899764] Key type lgssc unregistered [12112.395863] LNet: 326883:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12112.405728] LNetError: 326883:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12112.417781] LNet: Removed LNI 192.168.202.110@tcp [12113.502690] Key type .llcrypt unregistered [12113.505533] Key type ._llcrypt unregistered [12140.870507] Key type ._llcrypt registered [12140.875458] Key type .llcrypt registered [12141.023971] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [12142.886946] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12142.964316] alg: No test for adler32 (adler32-zlib) [12144.212832] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [12144.537730] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [12146.271631] Key type lgssc registered [12147.721763] Lustre: Echo OBD driver; http://www.lustre.org/ [12156.741137] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12156.757600] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12158.222099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12163.753660] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12168.690508] LustreError: 328304:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12168.725714] LustreError: 328304:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12172.689901] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12173.794068] LustreError: 328303:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12178.915587] LustreError: 328304:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12180.842249] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12181.351658] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12186.513168] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12198.232964] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12198.689731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12203.820177] LustreError: 329983:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12203.847616] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12205.897488] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12212.203739] LustreError: 329985:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12212.231625] LustreError: 329985:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12214.791864] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12222.397743] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12222.437884] LustreError: 329986:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12222.468436] LustreError: 329986:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12222.712605] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12228.109026] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12231.021256] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12239.819075] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12249.522278] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12249.779438] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12361.184853] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12361.190564] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12361.200733] LustreError: Skipped 1 previous similar message [12361.205282] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12361.239943] Lustre: Skipped 1 previous similar message [12362.966114] Lustre: server umount lustre-OST0001 complete [12366.319761] LustreError: 329986:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12366.348174] LustreError: 329986:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12370.912916] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12370.929059] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12370.947796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12370.959542] Lustre: Skipped 1 previous similar message [12376.773940] Lustre: server umount lustre-OST0000 complete [12386.277069] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12386.280548] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12386.288175] Lustre: Skipped 1 previous similar message [12386.295943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12386.298233] Lustre: Skipped 2 previous similar messages [12388.916786] Lustre: server umount lustre-MDT0000 complete [12393.952187] LustreError: 328285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873184 with bad export cookie 17376623396282246853 [12393.963115] LustreError: 328285:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12393.969650] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12394.539038] Lustre: server umount lustre-MDT0001 complete [12411.488957] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [12413.893637] Key type lgssc unregistered [12414.081972] LNet: 333279:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12414.085742] LNetError: 333279:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12414.093797] LNet: Removed LNI 192.168.202.110@tcp [12414.950054] Key type .llcrypt unregistered [12414.951453] Key type ._llcrypt unregistered [12462.348273] Key type ._llcrypt registered [12462.349625] Key type .llcrypt registered [12462.575274] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [12464.496170] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12464.581932] alg: No test for adler32 (adler32-zlib) [12465.743706] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [12466.076795] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [12467.815162] Key type lgssc registered [12469.764788] Lustre: Echo OBD driver; http://www.lustre.org/ [12483.349759] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12483.361678] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12484.851006] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12490.175568] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12494.306422] LustreError: 335571:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12494.317686] LustreError: 335571:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12499.432439] LustreError: 335570:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12501.146838] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12501.685541] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12507.623244] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12511.372300] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12520.067965] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12520.395944] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12526.513143] LustreError: 337061:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12528.370341] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12529.603012] LustreError: 337059:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12529.629552] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12529.638934] LustreError: 337059:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12534.765963] LustreError: 337061:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12534.789508] LustreError: 337061:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12538.635151] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12538.918891] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12544.008100] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12547.362186] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12558.516078] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12566.293374] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12579.826179] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12579.853430] Lustre: Skipped 2 previous similar messages [12579.858719] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12579.878671] Lustre: Skipped 3 previous similar messages [12584.932299] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12584.950204] Lustre: Skipped 3 previous similar messages [12586.340983] Lustre: server umount lustre-MDT0000 complete [12590.054981] LustreError: 335565:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12590.068363] LustreError: 335565:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12590.845432] LustreError: 335550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873381 with bad export cookie 4644371488213562239 [12590.854377] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12590.864239] LustreError: 335550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12591.284201] Lustre: server umount lustre-MDT0001 complete [12606.612965] Lustre: server umount lustre-OST0000 complete [12621.656766] Lustre: server umount lustre-OST0001 complete [12641.699342] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [12653.100198] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12653.582636] LustreError: 340837:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12653.723236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12659.037152] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12670.144418] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12670.708748] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12676.316372] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12680.044273] Lustre: 341978:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12688.306901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12688.600834] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12695.726706] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12695.811230] LustreError: 342331:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12695.849627] LustreError: 342331:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12698.884826] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12705.518292] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12710.948598] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12713.264202] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [12721.567766] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12725.703787] Lustre: 343850:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12731.877710] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12731.891883] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12731.900413] Lustre: Skipped 1 previous similar message [12731.905597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12734.432941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12734.452560] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12734.479876] Lustre: Skipped 2 previous similar messages [12736.641400] Lustre: server umount lustre-MDT0000 complete [12740.608260] LustreError: 340818:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873531 with bad export cookie 4644371488213563940 [12740.629213] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12741.122902] Lustre: server umount lustre-MDT0001 complete [12756.573718] Lustre: server umount lustre-OST0000 complete [12770.979204] Lustre: server umount lustre-OST0001 complete [12774.929438] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12777.135716] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12779.566859] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 19:32:49 (1787873569) [12807.382832] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [12816.609220] loop0: detected capacity change from 0 to 1099511627776 [12817.181250] SGI XFS with ACLs, security attributes, quota, no debug enabled [12817.190118] *** VALIDATE xfs *** [12817.197802] XFS (loop0): Mounting V5 Filesystem [12817.219853] XFS (loop0): Ending clean mount [12818.412197] loop1: detected capacity change from 0 to 18691697672192 [12937.751807] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [12949.015915] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [12989.436452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12989.760219] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12989.806771] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12989.874745] Lustre: lustre-MDT0000: new disk, initializing [12989.950842] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12989.955595] Lustre: Skipped 1 previous similar message [12989.978979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12996.493190] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13009.436599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13009.576978] Lustre: 348989:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13009.684699] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13009.703735] Lustre: Skipped 1 previous similar message [13009.786528] Lustre: lustre-MDT0001: new disk, initializing [13009.914479] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13009.922319] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13013.740409] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13024.789122] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13032.285622] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13040.808174] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13041.082612] Lustre: lustre-OST0000: new disk, initializing [13041.085310] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13041.091779] Lustre: 350960:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13042.371895] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13042.388358] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13042.535766] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13048.757539] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13059.199525] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13064.629700] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13064.842216] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13070.386859] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13070.737336] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13073.390179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13073.398334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13073.406522] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13073.406533] Lustre: Skipped 3 previous similar messages [13073.443212] LustreError: Skipped 1 previous similar message [13079.402405] Lustre: server umount lustre-OST0000 complete [13086.690358] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13086.702303] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13086.714580] Lustre: Skipped 1 previous similar message [13086.719217] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13086.723481] Lustre: Skipped 3 previous similar messages [13090.442231] Lustre: server umount lustre-MDT0000 complete [13093.858731] LustreError: 349001:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13093.866554] LustreError: 349001:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 9 previous similar messages [13094.226517] LustreError: 348980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873885 with bad export cookie 4644371488213565487 [13094.228028] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13094.235856] LustreError: 348980:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13094.734946] Lustre: server umount lustre-MDT0001 complete [13102.902974] XFS (loop0): Unmounting Filesystem [13107.803677] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 19:38:18 (1787873898) [13115.760383] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13116.445851] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13116.458702] Lustre: Skipped 2 previous similar messages [13121.096437] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13131.213990] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13137.046812] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13145.183543] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13151.158678] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13158.289450] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13165.473325] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13173.586001] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13187.556127] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13187.561847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13187.570153] Lustre: Skipped 2 previous similar messages [13187.574854] Lustre: Skipped 2 previous similar messages [13187.591229] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13191.999430] Lustre: server umount lustre-OST0000 complete [13203.037107] Lustre: server umount lustre-MDT0000 complete [13206.860253] LustreError: 353503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787873997 with bad export cookie 4644371488213566369 [13206.872450] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13206.884346] LustreError: 353503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13207.220930] Lustre: server umount lustre-MDT0001 complete [13218.535662] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [13221.549263] Key type lgssc unregistered [13221.857750] LNet: 357157:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13221.869722] LNetError: 357157:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13221.887765] LNet: Removed LNI 192.168.202.110@tcp [13222.668326] Key type .llcrypt unregistered [13222.671885] Key type ._llcrypt unregistered [13256.439335] Key type ._llcrypt registered [13256.443128] Key type .llcrypt registered [13256.509803] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [13258.202804] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13258.218717] alg: No test for adler32 (adler32-zlib) [13259.305579] Lustre: Lustre: Build Version: 2.17.57_120_gb73d7ca [13259.520398] LNet: Added LNI 192.168.202.110@tcp [8/256/0/180] [13261.399492] Key type lgssc registered [13262.511805] Lustre: Echo OBD driver; http://www.lustre.org/ [13270.713337] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13270.730413] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13272.074709] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13276.156047] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13282.277289] LustreError: 358729:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13282.311690] LustreError: 358729:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13286.327015] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13287.059436] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13292.428652] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13301.994677] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13308.833257] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13316.709843] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13317.098511] LustreError: 360550:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13317.265611] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13324.459290] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13332.902721] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13342.189565] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13342.214869] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13342.220879] Lustre: Skipped 1 previous similar message [13345.090044] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.10@tcp (stopping) [13346.928978] Lustre: server umount lustre-MDT0001 complete [13347.302392] LustreError: 358724:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13347.311909] LustreError: 358724:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [13350.194787] LustreError: 358725:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.10@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13355.307275] LustreError: 361172:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.10@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13355.331961] LustreError: 361172:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13359.075262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13359.148404] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13359.150341] Lustre: Found index 1 for lustre-MDT0001, updating log [13359.171206] Lustre: 358716:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13359.473684] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13364.157966] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13364.726858] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13366.768978] LustreError: 362312:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13366.772446] LustreError: 362312:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13366.775933] LustreError: 358721:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13366.815354] LustreError: 362322:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.110@tcp: cfg command failed: rc = -17 [13366.820411] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13366.820411] [13374.945854] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13374.954354] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13374.968354] Lustre: Skipped 1 previous similar message [13374.976780] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13378.848582] Lustre: server umount lustre-MDT0001 complete [13380.070856] LustreError: 359458:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13380.087835] LustreError: 359458:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [13389.601418] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13389.796449] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13389.803065] Lustre: Found index 1 for lustre-MDT0001, updating log [13389.865275] Lustre: 358717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13390.226792] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13394.962708] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13395.458499] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13400.052483] LustreError: 363417:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13400.061326] LustreError: 363417:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13400.067342] LustreError: 358721:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13400.074980] LustreError: 358721:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13400.136599] LustreError: 363420:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13400.144574] LustreError: 363420:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13400.158570] LustreError: 363420:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.110@tcp: cfg command failed: rc = -17 [13400.172860] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.110@tcp [13400.172860] [13702.624936] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13702.628772] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13702.636301] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13702.655091] Lustre: Skipped 1 previous similar message [13702.659908] Lustre: Skipped 2 previous similar messages [13707.748606] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13707.754836] Lustre: Skipped 2 previous similar messages [13714.399117] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13714.723766] Lustre: server umount lustre-MDT0001 complete [13717.991883] LustreError: 360815:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13718.014947] LustreError: 360815:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [13727.000514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13727.250737] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13727.268790] Lustre: Found index 1 for lustre-MDT0001, updating log [13727.417821] Lustre: 358716:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13728.024583] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13732.255643] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [13733.352445] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13736.973391] LustreError: 371890:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13736.979479] LustreError: 371890:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13737.008519] LustreError: 358721:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13737.023536] LustreError: 358721:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13737.075446] LustreError: 371893:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13737.098313] LustreError: 371893:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.110@tcp: cfg command failed: rc = -114 [13737.118588] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13737.118588] [14055.398072] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14055.408374] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14055.424110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14055.430757] Lustre: Skipped 5 previous similar messages [14061.040121] LustreError: 358728:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14061.057124] LustreError: 358728:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [14061.231495] Lustre: server umount lustre-MDT0000 complete [14065.438893] LustreError: 358709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787874856 with bad export cookie 837173516688983298 [14065.443318] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14065.459068] LustreError: 358709:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14066.146857] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14066.168894] Lustre: Skipped 2 previous similar messages [14072.232648] Lustre: server umount lustre-MDT0001 complete [14076.516964] Lustre: server umount lustre-OST0000 complete [14087.695039] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [14098.766851] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [14138.894905] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 19:55:29 (1787874929) [14146.673667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14146.985620] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14147.018552] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14147.084484] Lustre: lustre-MDT0000: new disk, initializing [14147.165645] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14147.186178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14152.103454] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [14166.223700] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14166.406919] Lustre: 382265:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14166.546096] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14166.555451] Lustre: Skipped 1 previous similar message [14166.797408] Lustre: lustre-MDT0001: new disk, initializing [14166.965973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14167.015587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14167.032183] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14172.521511] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [14184.210139] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14191.780541] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14199.649199] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14199.935513] Lustre: lustre-OST0000: new disk, initializing [14199.945150] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14199.957779] Lustre: 384234:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14200.105541] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14205.524498] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14205.537365] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14205.669796] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14207.242232] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_default_debug -1 all [14218.610773] Lustre: DEBUG MARKER: oleg210-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14231.530862] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14231.535124] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14231.554451] Lustre: Skipped 2 previous similar messages [14231.573057] Lustre: Skipped 6 previous similar messages [14233.572517] Lustre: server umount lustre-MDT0000 complete [14236.645735] LustreError: 382272:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14236.666759] LustreError: 382272:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [14237.288182] LustreError: 382258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787875028 with bad export cookie 837173516688985258 [14237.290580] LustreError: MGC192.168.202.110@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14237.295975] LustreError: 382258:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14237.546625] Lustre: server umount lustre-MDT0001 complete [14251.442871] Lustre: server umount lustre-OST0000 complete [14283.216973] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing set_hostid [14292.788423] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing load_modules_local [14332.028987] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14005 sec ========= 19:58:42 (1787875122) [14334.263493] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 19:58:44 (1787875124) === [14338.193257] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 19:58:48 (1787875128) === [14384.175905] Lustre: DEBUG MARKER: oleg210-server.virtnet: executing unload_modules_local [14387.364538] Key type lgssc unregistered [14387.718929] LNet: 390606:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14387.733979] LNetError: 390606:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14388.776770] LNet: Removed LNI 192.168.202.110@tcp [14390.046630] Key type .llcrypt unregistered [14390.053505] Key type ._llcrypt unregistered