[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 528458964 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002379] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004013] kvm-guest: setup PV IPIs [ 0.006884] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008013] pid_max: default: 32768 minimum: 301 [ 0.010162] LSM: Security Framework initializing [ 0.011070] Yama: becoming mindful. [ 0.012043] SELinux: Initializing. [ 0.013079] *** VALIDATE selinux *** [ 0.022019] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027384] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028228] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030123] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032014] *** VALIDATE tmpfs *** [ 0.033567] *** VALIDATE proc *** [ 0.035360] *** VALIDATE cgroup *** [ 0.036017] *** VALIDATE cgroup2 *** [ 0.038169] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039208] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040020] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041047] Spectre V2 : User space: Vulnerable [ 0.042013] Speculative Store Bypass: Vulnerable [ 0.045960] debug: unmapping init [mem 0xffffffffb6059000-0xffffffffb6060fff] [ 0.047183] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048700] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049027] ... version: 2 [ 0.050012] ... bit width: 48 [ 0.051014] ... generic registers: 4 [ 0.052012] ... value mask: 0000ffffffffffff [ 0.053014] ... max period: 00007fffffffffff [ 0.054014] ... fixed-purpose events: 3 [ 0.055012] ... event mask: 000000070000000f [ 0.056326] rcu: Hierarchical SRCU implementation. [ 0.058447] smp: Bringing up secondary CPUs ... [ 0.059590] x86: Booting SMP configuration: [ 0.060025] .... node #0, CPUs: #1 #2 #3 [ 0.063389] smp: Brought up 1 node, 4 CPUs [ 0.065014] smpboot: Max logical packages: 1 [ 0.066018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.132018] node 0 deferred pages initialised in 64ms [ 0.135403] devtmpfs: initialized [ 0.136231] x86/mm: Memory block size: 128MB [ 0.138539] gcov: version magic: 0x41383552 [ 0.140271] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.142103] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.144281] pinctrl core: initialized pinctrl subsystem [ 0.145123] [ 0.145510] ************************************************************* [ 0.147010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.149009] ** ** [ 0.151010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.153014] ** ** [ 0.155013] ** This means that this kernel is built to expose internal ** [ 0.157013] ** IOMMU data structures, which may compromise security on ** [ 0.160012] ** your system. ** [ 0.162011] ** ** [ 0.163009] ** If you see this message and you are not debugging the ** [ 0.165013] ** kernel, report this immediately to your vendor! ** [ 0.168011] ** ** [ 0.170012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.172013] ************************************************************* [ 0.174634] NET: Registered protocol family 16 [ 0.176278] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.178053] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.180060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.183151] cpuidle: using governor menu [ 0.184805] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.188464] PCI: Using configuration type 1 for base access [ 0.190144] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.198092] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.201030] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.203147] cryptd: max_cpu_qlen set to 1000 [ 0.206240] ACPI: Added _OSI(Module Device) [ 0.208011] ACPI: Added _OSI(Processor Device) [ 0.209010] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.211013] ACPI: Added _OSI(Processor Aggregator Device) [ 0.215254] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.221078] ACPI: Interpreter enabled [ 0.222048] ACPI: PM: (supports S0 S3 S4 S5) [ 0.223008] ACPI: Using IOAPIC for interrupt routing [ 0.224074] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.226338] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.235035] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.237033] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.239015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.241070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.244406] acpiphp: Slot [2] registered [ 0.246114] acpiphp: Slot [5] registered [ 0.247092] acpiphp: Slot [6] registered [ 0.248074] acpiphp: Slot [7] registered [ 0.249109] acpiphp: Slot [8] registered [ 0.250095] acpiphp: Slot [9] registered [ 0.252136] acpiphp: Slot [10] registered [ 0.253116] acpiphp: Slot [3] registered [ 0.255100] acpiphp: Slot [4] registered [ 0.257110] acpiphp: Slot [11] registered [ 0.258095] acpiphp: Slot [12] registered [ 0.260096] acpiphp: Slot [13] registered [ 0.261138] acpiphp: Slot [14] registered [ 0.263101] acpiphp: Slot [15] registered [ 0.264083] acpiphp: Slot [16] registered [ 0.265065] acpiphp: Slot [17] registered [ 0.266066] acpiphp: Slot [18] registered [ 0.267116] acpiphp: Slot [19] registered [ 0.269087] acpiphp: Slot [20] registered [ 0.270123] acpiphp: Slot [21] registered [ 0.271081] acpiphp: Slot [22] registered [ 0.272039] acpiphp: Slot [23] registered [ 0.273063] acpiphp: Slot [24] registered [ 0.274069] acpiphp: Slot [25] registered [ 0.275068] acpiphp: Slot [26] registered [ 0.276108] acpiphp: Slot [27] registered [ 0.278076] acpiphp: Slot [28] registered [ 0.279083] acpiphp: Slot [29] registered [ 0.280059] acpiphp: Slot [30] registered [ 0.281085] acpiphp: Slot [31] registered [ 0.282050] PCI host bridge to bus 0000:00 [ 0.283014] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.285037] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.286016] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.288024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.290016] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.293024] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.295127] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.298155] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.302473] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.310575] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.318071] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.320014] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.321011] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.324018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.326568] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.328755] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.331040] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.334358] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.338013] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.349014] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.353973] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.359644] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.367016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.376025] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.398020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.409039] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.415017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.425018] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.447015] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.458258] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.467026] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.479019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.507016] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.525826] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.540016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.555021] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.596019] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.611105] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.627022] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.646018] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.692017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.712234] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.723016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.737016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.776022] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.797000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.799342] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.801507] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.804367] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.806255] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.811101] iommu: Default domain type: Passthrough [ 0.813380] SCSI subsystem initialized [ 0.814109] ACPI: bus type USB registered [ 0.815113] usbcore: registered new interface driver usbfs [ 0.817060] usbcore: registered new interface driver hub [ 0.818066] usbcore: registered new device driver usb [ 0.819154] pps_core: LinuxPPS API ver. 1 registered [ 0.821009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.823101] PTP clock support registered [ 0.825167] EDAC MC: Ver: 3.0.0 [ 0.827218] PCI: Using ACPI for IRQ routing [ 0.829142] NetLabel: Initializing [ 0.830014] NetLabel: domain hash size = 128 [ 0.832010] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.834107] NetLabel: unlabeled traffic allowed by default [ 0.836343] vgaarb: loaded [ 0.837786] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.839011] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.846143] clocksource: Switched to clocksource kvm-clock [ 0.955541] VFS: Disk quotas dquot_6.6.0 [ 0.956695] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.958578] *** VALIDATE ramfs *** [ 0.959539] *** VALIDATE hugetlbfs *** [ 0.960758] pnp: PnP ACPI init [ 0.962722] pnp: PnP ACPI: found 6 devices [ 0.977858] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.980770] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.982364] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.984074] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.985932] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.987925] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.990133] NET: Registered protocol family 2 [ 0.992295] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.996470] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.000804] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.006361] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.009791] TCP: Hash tables configured (established 65536 bind 65536) [ 1.013094] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.016634] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.019498] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.022554] NET: Registered protocol family 1 [ 1.028518] RPC: Registered named UNIX socket transport module. [ 1.030652] RPC: Registered udp transport module. [ 1.032358] RPC: Registered tcp transport module. [ 1.034392] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.036576] NET: Registered protocol family 44 [ 1.037916] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.039902] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.041711] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.043741] PCI: CLS 0 bytes, default 64 [ 1.045996] Unpacking initramfs... [ 2.510567] debug: unmapping init [mem 0xffff9caf3cc54000-0xffff9caf3ffbffff] [ 2.514054] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.516434] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.519502] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.009253] Initialise system trusted keyrings [ 3.010852] Key type blacklist registered [ 3.012847] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.022115] zbud: loaded [ 3.025025] *** VALIDATE nfs *** [ 3.026186] *** VALIDATE nfs4 *** [ 3.027606] pstore: using deflate compression [ 3.031022] Platform Keyring initialized [ 3.117245] NET: Registered protocol family 38 [ 3.118413] Key type asymmetric registered [ 3.119465] Asymmetric key parser 'x509' registered [ 3.120743] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.123099] io scheduler mq-deadline registered [ 3.124600] io scheduler kyber registered [ 3.125739] io scheduler bfq registered [ 3.127280] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.129338] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.131259] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.133852] ACPI: Power Button [PWRF] [ 3.138168] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.143750] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.161755] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.173032] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.198660] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.226783] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.255116] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.259231] Non-volatile memory driver v1.3 [ 3.261145] Linux agpgart interface v0.103 [ 3.295391] virtio_blk virtio1: [vda] 136552 512-byte logical blocks (69.9 MB/66.7 MiB) [ 3.298158] vda: detected capacity change from 0 to 69914624 [ 3.327018] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.329353] vdb: detected capacity change from 0 to 1073741824 [ 3.347077] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.349276] vdc: detected capacity change from 0 to 2621440000 [ 3.375262] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.377489] vdd: detected capacity change from 0 to 2621440000 [ 3.397619] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.399866] vde: detected capacity change from 0 to 4294967296 [ 3.417893] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.420195] vdf: detected capacity change from 0 to 4294967296 [ 3.426799] libphy: Fixed MDIO Bus: probed [ 3.431603] usbcore: registered new interface driver usbserial_generic [ 3.433795] usbserial: USB Serial support registered for generic [ 3.435790] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.439259] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.440554] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.442682] mousedev: PS/2 mouse device common for all mice [ 3.445698] rtc_cmos 00:05: RTC can wake from S4 [ 3.447935] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.450018] rtc_cmos 00:05: registered as rtc0 [ 3.456640] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.457062] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.462467] intel_pstate: CPU model not supported [ 3.466547] hid: raw HID events driver (C) Jiri Kosina [ 3.471943] usbcore: registered new interface driver usbhid [ 3.473302] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.473780] usbhid: USB HID core driver [ 3.476843] drop_monitor: Initializing network drop monitor service [ 3.478276] Initializing XFRM netlink socket [ 3.479489] NET: Registered protocol family 10 [ 3.481169] Segment Routing with IPv6 [ 3.481971] NET: Registered protocol family 17 [ 3.484648] mpls_gso: MPLS GSO support [ 3.489414] RAS: Correctable Errors collector initialized. [ 3.491109] AVX version of gcm_enc/dec engaged. [ 3.491981] AES CTR mode by8 optimization enabled [ 3.546611] sched_clock: Marking stable (3546591668, 0)->(4412544588, -865952920) [ 3.548774] registered taskstats version 1 [ 3.549935] Loading compiled-in X.509 certificates [ 3.551533] zswap: loaded using pool lzo/zbud [ 3.569339] Key type big_key registered [ 3.582414] Key type encrypted registered [ 3.583529] ima: No TPM chip found, activating TPM-bypass! [ 3.585176] ima: Allocated hash algorithm: sha1 [ 3.586719] ima: No architecture policies found [ 3.588338] evm: Initialising EVM extended attributes: [ 3.590249] evm: security.selinux [ 3.591505] evm: security.ima [ 3.592371] evm: security.capability [ 3.593437] evm: HMAC attrs: 0x1 [ 3.595778] rtc_cmos 00:05: setting system clock to 2026-06-12 00:40:10 UTC (1781224810) [ 3.601435] debug: unmapping init [mem 0xffffffffb7003000-0xffffffffb71fffff] [ 3.604385] debug: unmapping init [mem 0xffffffffb5d82000-0xffffffffb6058fff] [ 3.613132] Write protecting the kernel read-only data: 28672k [ 3.615400] debug: unmapping init [mem 0xffffffffb4403000-0xffffffffb45fffff] [ 3.616920] debug: unmapping init [mem 0xffffffffb4d14000-0xffffffffb4dfffff] [ 3.649316] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.654448] systemd[1]: Detected virtualization kvm. [ 3.655582] systemd[1]: Detected architecture x86-64. [ 3.656980] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.681253] systemd[1]: No hostname configured. [ 3.682223] systemd[1]: Set hostname to . [ 3.683423] random: systemd: uninitialized urandom read (16 bytes read) [ 3.685139] systemd[1]: Initializing machine ID from random generator. [ 3.750035] random: ln: uninitialized urandom read (6 bytes read) [ 3.823837] random: systemd: uninitialized urandom read (16 bytes read) [ 3.825584] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 3.831670] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 3.834110] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Journal Service... [ OK ] Reached target Sockets. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Reached target Slices. Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.419358] device-mapper: uevent: version 1.0.3 [ 4.421441] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 5.077791] virtio_net virtio0 ens2: renamed from eth0 [ 5.085586] random: fast init done Starting dracut initqueue hook... [ 5.197709] scsi host0: ata_piix [ 5.205367] scsi host1: ata_piix [ 5.206460] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.208111] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 8.844867] dracut-initqueue[580]: RTNETLINK answers: File exists [ 10.097949] random: crng init done [ 10.100574] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.580349] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.811924] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.093453] SELinux: Disabled at runtime. [ 12.165243] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.175721] systemd[1]: Detected virtualization kvm. [ 12.177500] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.737498] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.740021] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.746169] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.749887] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.753371] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.765379] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.769893] systemd[1]: Stopped target Switch Root. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Starting Apply Kernel Variables... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Kernel Debug File System... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Initrd File Systems. [ 12.876322] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Reached target rpc_pipefs.target. [ OK ] Reached target Paths. [ OK ] Listening on RPCbind Server Activation Socket. Mounting POSIX Message Queue File System... [ OK ] Reached target RPC Port Mapper. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Started Journal Service. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 13.272373] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.540946] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.583866] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.798646] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.813287] EDAC sbridge: Ver: 1.1.2 [ 15.747437] Key type dns_resolver registered [ 16.153517] NFS: Registering the id_resolver key type [ 16.156117] Key type id_resolver registered [ 16.157909] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started D-Bus System Message Bus. [ OK ] Started dnf makecache --timer. [ OK ] Started irqbalance daemon. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Getty on tty1. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg136-server login: [ 57.397631] libcfs: loading out-of-tree module taints kernel. [ 57.448836] Key type ._llcrypt registered [ 57.451889] Key type .llcrypt registered [ 57.654612] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 81.442738] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 83.761301] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 83.791422] alg: No test for adler32 (adler32-zlib) [ 85.678557] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 85.956007] hrtimer: interrupt took 6594406 ns [ 86.761778] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 88.727256] Key type lgssc registered [ 90.961291] Lustre: Echo OBD driver; http://www.lustre.org/ [ 112.534772] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 157.049938] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 175.375908] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 175.461099] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 177.081797] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 177.126139] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 177.224105] Lustre: lustre-MDT0000: new disk, initializing [ 177.391207] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 177.460121] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 183.936492] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 199.420026] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 199.574835] Lustre: 6505:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 199.648085] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 199.653868] Lustre: Skipped 1 previous similar message [ 199.834303] Lustre: lustre-MDT0001: new disk, initializing [ 199.995493] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 200.043349] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 200.059250] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 204.605542] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.725331] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 220.042688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 220.386094] Lustre: lustre-OST0000: new disk, initializing [ 220.394332] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 220.399160] Lustre: 8410:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 220.483070] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 226.332527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 226.342184] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 226.470648] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 226.677658] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 242.877937] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 243.053502] Lustre: lustre-OST0001: new disk, initializing [ 243.057628] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 243.062321] Lustre: 9467:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 243.127307] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 250.779611] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 252.499455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 252.511937] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 252.581088] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 263.513228] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 272.728760] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 279.999975] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing check_logdir /tmp/testlogs/ [ 285.444672] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing yml_node [ 289.575611] Lustre: DEBUG MARKER: Client: 2.17.53.70 [ 292.733420] Lustre: DEBUG MARKER: MDS: 2.17.53.70 [ 295.514895] Lustre: DEBUG MARKER: OSS: 2.17.53.70 [ 297.326578] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jun 11 20:45:02 EDT 2026 [ 320.752910] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 322.960385] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 339.424898] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 339.442230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 339.454872] Lustre: Skipped 1 previous similar message [ 339.484587] Lustre: Skipped 3 previous similar messages [ 344.544383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 344.550286] Lustre: Skipped 3 previous similar messages [ 344.987694] Lustre: server umount lustre-MDT0000 complete [ 348.977464] LustreError: 6497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225155 with bad export cookie 8106202938980968852 [ 348.982820] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 348.987665] LustreError: 6497:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 349.668159] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 349.674108] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 349.687034] Lustre: Skipped 3 previous similar messages [ 349.698360] Lustre: Skipped 1 previous similar message [ 353.696343] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 353.700314] Lustre: Skipped 1 previous similar message [ 355.530961] Lustre: server umount lustre-MDT0001 complete [ 360.102313] Lustre: server umount lustre-OST0000 complete [ 364.555637] Lustre: server umount lustre-OST0001 complete [ 371.912929] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 381.616778] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 424.158309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 424.572284] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 424.650425] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 424.750832] Lustre: lustre-MDT0000: new disk, initializing [ 424.812621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 424.838293] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 429.517678] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 441.150302] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 441.296068] Lustre: 14986:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 441.391458] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 441.397465] Lustre: Skipped 1 previous similar message [ 441.536360] Lustre: lustre-MDT0001: new disk, initializing [ 441.694647] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 441.810038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 441.820269] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 447.013736] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 455.158974] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 459.789740] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 467.537576] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 467.751085] Lustre: lustre-OST0000: new disk, initializing [ 467.755964] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 467.767658] Lustre: 16727:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 467.906767] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 469.025265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 469.033248] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 469.216269] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 473.918639] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 481.976128] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 487.231666] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 487.475633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 492.966531] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 493.227543] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 500.195080] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 500.220694] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 500.228875] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 501.794906] Lustre: server umount lustre-OST0000 complete [ 508.398778] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 508.408091] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 508.423528] Lustre: Skipped 1 previous similar message [ 508.429175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 508.439599] Lustre: Skipped 1 previous similar message [ 512.523966] Lustre: server umount lustre-MDT0000 complete [ 515.552684] LustreError: 14996:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 515.573682] LustreError: 14996:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 516.665135] LustreError: 14976:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225323 with bad export cookie 8106202938980970385 [ 516.682499] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 517.082485] Lustre: server umount lustre-MDT0001 complete [ 523.345919] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 20:48:48 (1781225328) [ 531.409513] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 532.036501] LustreError: 18846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 532.174120] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 537.024662] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 537.579138] LustreError: 18847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 541.687117] LustreError: 18846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 546.787558] LustreError: 18847:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 547.014223] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 552.448574] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 559.017569] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 563.551437] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 571.256432] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 579.573789] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 586.473119] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 590.435570] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 597.486256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 597.497274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 597.497732] Lustre: Skipped 2 previous similar messages [ 597.530087] Lustre: Skipped 4 previous similar messages [ 603.208034] Lustre: server umount lustre-MDT0000 complete [ 607.307239] LustreError: 20463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225414 with bad export cookie 8106202938980971267 [ 607.310251] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 607.326917] LustreError: 20463:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 607.622950] Lustre: server umount lustre-MDT0001 complete [ 621.515546] Lustre: server umount lustre-OST0000 complete [ 645.937707] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 658.040374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 658.421775] LustreError: 22870:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 658.527782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 658.536268] Lustre: Skipped 2 previous similar messages [ 663.779866] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 673.661411] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 679.745386] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 682.741731] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 689.689285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 698.078855] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 708.032952] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 708.242240] Lustre: lustre-OST0001: new disk, initializing [ 708.246211] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 708.251171] Lustre: 25133:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 710.463934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 710.490617] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 710.691217] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 715.557703] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 726.256630] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 730.944801] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 751.079576] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 751.091456] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 751.114912] Lustre: Skipped 1 previous similar message [ 751.122938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 751.128049] Lustre: Skipped 3 previous similar messages [ 752.825372] Lustre: server umount lustre-MDT0000 complete [ 756.709046] LustreError: 25648:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 756.751092] LustreError: 25648:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 756.929621] LustreError: 26087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225563 with bad export cookie 8106202938980972114 [ 756.950900] LustreError: 26087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 756.958086] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 757.522826] Lustre: server umount lustre-MDT0001 complete [ 771.359720] Lustre: server umount lustre-OST0000 complete [ 786.193560] Lustre: server umount lustre-OST0001 complete [ 806.646961] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 818.608243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 818.985304] LustreError: 28123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 819.144300] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 819.161197] Lustre: Skipped 3 previous similar messages [ 824.721375] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 834.803668] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 840.401720] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 843.345934] Lustre: 29236:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 851.555054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 859.406365] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 869.346499] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 877.307200] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 885.656820] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 890.376419] Lustre: 31082:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 896.994250] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 896.998873] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 897.008894] Lustre: Skipped 3 previous similar messages [ 897.017822] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 897.022080] Lustre: Skipped 3 previous similar messages [ 903.299770] Lustre: server umount lustre-MDT0000 complete [ 905.714249] LustreError: 28124:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 905.744665] LustreError: 28124:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 907.424372] LustreError: 28103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225714 with bad export cookie 8106202938980973409 [ 907.435529] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 907.437048] LustreError: 28103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 907.830468] Lustre: server umount lustre-MDT0001 complete [ 923.149291] Lustre: server umount lustre-OST0000 complete [ 926.184726] Lustre: 3655:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781225717/real 1781225717] req@ffff9caf8b2e1e00 x1867749673708160/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781225733 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 927.644646] Lustre: server umount lustre-OST0001 complete [ 937.901790] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 20:55:43 (1781225743) [ 968.707988] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 979.450541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 985.544701] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 995.446744] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1001.598921] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1005.114268] Lustre: 34989:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1012.655319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1021.846317] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1031.227327] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1038.484526] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1046.625776] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1050.986524] Lustre: 36832:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1057.253819] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1057.273951] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1057.275275] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1057.275282] Lustre: Skipped 4 previous similar messages [ 1057.295067] Lustre: Skipped 5 previous similar messages [ 1062.282773] Lustre: server umount lustre-MDT0000 complete [ 1062.386037] LustreError: 33864:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1062.434474] LustreError: 33864:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 12 previous similar messages [ 1066.659416] LustreError: 33849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781225873 with bad export cookie 8106202938980974515 [ 1066.665300] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1066.673918] LustreError: 33849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1067.137345] Lustre: server umount lustre-MDT0001 complete [ 1082.062053] Lustre: server umount lustre-OST0000 complete [ 1083.679708] Lustre: 3654:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781225874/real 1781225874] req@ffff9cafb7d3b840 x1867749673760896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781225890 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1087.216602] Lustre: server umount lustre-OST0001 complete [ 1098.549411] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 20:58:23 (1781225903) [ 1116.251958] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1127.290799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1127.877612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1127.889186] Lustre: Skipped 7 previous similar messages [ 1132.633285] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1142.786061] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1148.823281] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1151.858827] Lustre: 40071:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1159.524284] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1166.746518] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1176.628077] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1183.324409] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1192.280755] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1196.801858] Lustre: 41914:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1197.862559] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1197.872102] Lustre: Skipped 1 previous similar message [ 1202.667266] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1202.694466] Lustre: Skipped 4 previous similar messages [ 1208.150532] Lustre: server umount lustre-MDT0000 complete [ 1212.775559] LustreError: 38938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781226019 with bad export cookie 8106202938980975621 [ 1212.780410] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1212.802794] LustreError: 38938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1227.231173] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1227.642145] Lustre: server umount lustre-MDT0001 complete [ 1238.386672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1263.853421] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1272.823255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1278.362480] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1282.535736] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1285.050538] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1287.151227] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1291.802942] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1293.651382] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1306.094325] Lustre: server umount lustre-MDT0000 complete [ 1309.512830] LustreError: 39692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781226116 with bad export cookie 8106202938980976748 [ 1309.517421] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1309.522795] LustreError: 39692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1309.873651] Lustre: server umount lustre-MDT0001 complete [ 1323.278848] Lustre: server umount lustre-OST0000 complete [ 1336.453949] Lustre: server umount lustre-OST0001 complete [ 1345.371463] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 21:02:31 (1781226151) [ 1363.008814] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1376.398755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1376.918714] LustreError: 46300:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1376.942451] LustreError: 46300:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 27 previous similar messages [ 1381.307441] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1392.150428] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1398.944971] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1402.515710] Lustre: 47416:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1409.441948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1416.646819] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1425.002981] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1431.159733] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1438.926685] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1443.879698] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1458.554864] Lustre: server umount lustre-MDT0000 complete [ 1459.182799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1459.195405] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1459.214765] Lustre: Skipped 6 previous similar messages [ 1462.503478] LustreError: 46281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781226269 with bad export cookie 8106202938980977539 [ 1462.507536] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1462.514703] LustreError: 46281:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1462.933410] Lustre: server umount lustre-MDT0001 complete [ 1477.230045] Lustre: server umount lustre-OST0000 complete [ 1491.827162] Lustre: server umount lustre-OST0001 complete [ 1501.532932] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 21:05:07 (1781226307) [ 1503.987087] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1506.181154] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 21:05:11 (1781226311) [ 1518.131455] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 1527.476785] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1548.570370] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1549.142071] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1549.219879] Lustre: lustre-MDT0000: new disk, initializing [ 1549.374901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1554.761660] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1563.318833] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1570.863683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1571.100503] Lustre: lustre-OST0000: new disk, initializing [ 1571.113250] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1571.119060] Lustre: Skipped 1 previous similar message [ 1571.128796] Lustre: 53309:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1572.941867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1572.960786] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1573.012051] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1577.857908] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1587.108076] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1589.022332] Lustre: DEBUG MARKER: create test files [ 1618.914304] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1618.919314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1618.922182] Lustre: Skipped 27 previous similar messages [ 1622.671429] Lustre: server umount lustre-OST0000 complete [ 1627.132128] Lustre: server umount lustre-MDT0000 complete [ 1641.180582] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 1645.813724] Key type lgssc unregistered [ 1646.373132] LNet: 54807:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1646.388536] LNetError: 54807:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1647.412488] LNet: Removed LNI 192.168.201.136@tcp [ 1649.028719] Key type .llcrypt unregistered [ 1649.030393] Key type ._llcrypt unregistered [ 1682.261654] Key type ._llcrypt registered [ 1682.264233] Key type .llcrypt registered [ 1682.364380] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1683.627649] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1683.642561] alg: No test for adler32 (adler32-zlib) [ 1684.851426] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 1685.226629] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 1686.903172] Key type lgssc registered [ 1688.097457] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1696.145659] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1696.185944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1697.807756] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1703.731744] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1709.726371] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1717.026088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1717.368909] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1722.492275] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1722.632530] Lustre: 57419:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1722.642272] Lustre: 57419:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1722.648044] Lustre: 57419:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1722.670757] Lustre: 57419:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1722.681289] Lustre: 57419:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1722.694628] Lustre: 57419:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1724.564446] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1731.078469] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1736.864809] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1742.878206] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1752.581400] Lustre: server umount lustre-OST0000 complete [ 1766.662384] Lustre: server umount lustre-MDT0000 complete [ 1777.621364] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 1780.524539] Key type lgssc unregistered [ 1780.835110] LNet: 58901:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1780.842853] LNetError: 58901:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1780.859699] LNet: Removed LNI 192.168.201.136@tcp [ 1782.108517] Key type .llcrypt unregistered [ 1782.115237] Key type ._llcrypt unregistered [ 1815.845751] Key type ._llcrypt registered [ 1815.849675] Key type .llcrypt registered [ 1815.949524] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1816.624710] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1816.753822] alg: No test for adler32 (adler32-zlib) [ 1817.990866] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 1818.153789] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 1819.791654] Key type lgssc registered [ 1821.356788] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1833.325675] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1833.348719] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1835.472484] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1840.837973] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1845.884537] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1855.286480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1855.719390] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1860.849101] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1860.947204] Lustre: 61609:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1860.958027] Lustre: 61609:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1860.964799] Lustre: 61609:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1860.977275] Lustre: 61609:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1860.994323] Lustre: 61609:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1860.999180] Lustre: 61609:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1862.203968] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 1868.461190] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1874.320400] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1886.691954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1886.704843] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1889.159861] Lustre: server umount lustre-OST0000 complete [ 1892.603704] Lustre: server umount lustre-MDT0000 complete [ 1905.882438] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 1909.426735] Key type lgssc unregistered [ 1909.836504] LNet: 63037:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1909.843696] LNetError: 63037:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1909.855723] LNet: Removed LNI 192.168.201.136@tcp [ 1910.892642] Key type .llcrypt unregistered [ 1910.896738] Key type ._llcrypt unregistered [ 1930.177838] Key type ._llcrypt registered [ 1930.180435] Key type .llcrypt registered [ 1930.363281] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 1950.574410] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 1952.413635] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1952.723978] alg: No test for adler32 (adler32-zlib) [ 1953.963419] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 1954.332578] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 1956.063334] Key type lgssc registered [ 1957.403176] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2005.769634] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2005.893692] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2007.455638] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2007.501266] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2007.676271] Lustre: lustre-MDT0000: new disk, initializing [ 2007.832654] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2007.884130] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2014.776721] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2027.678717] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2027.850632] Lustre: 66158:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2027.932988] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2027.943964] Lustre: Skipped 1 previous similar message [ 2028.038135] Lustre: lustre-MDT0001: new disk, initializing [ 2028.132570] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2028.168522] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2028.189858] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2033.490693] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2041.435467] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2045.671790] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2053.040260] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2053.310289] Lustre: lustre-OST0000: new disk, initializing [ 2053.315087] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2053.322730] Lustre: 67905:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2053.450872] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2060.448247] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2063.509969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2063.532497] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2063.699589] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2068.124731] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2072.765064] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2072.978771] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2077.513093] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2077.667450] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2084.340513] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2084.345971] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2084.353440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2084.364724] Lustre: Skipped 1 previous similar message [ 2084.381427] Lustre: Skipped 1 previous similar message [ 2085.812451] Lustre: server umount lustre-OST0000 complete [ 2089.440313] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2089.441791] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2089.446022] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2089.474554] Lustre: Skipped 1 previous similar message [ 2094.561306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2094.567401] Lustre: Skipped 2 previous similar messages [ 2095.781806] Lustre: server umount lustre-MDT0000 complete [ 2099.604391] LustreError: 66149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781226906 with bad export cookie 1971340413687023908 [ 2099.614236] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2099.620095] LustreError: 66149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2099.907800] Lustre: server umount lustre-MDT0001 complete [ 2108.700260] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 21:15:14 (1781226914) [ 2121.496515] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 2167.121932] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 2179.071039] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 2221.651390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2222.238989] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2222.312619] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2222.472338] Lustre: lustre-MDT0000: new disk, initializing [ 2222.621278] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2222.642738] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2227.892745] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2239.902459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2240.024381] Lustre: 73726:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2240.085377] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2240.089207] Lustre: Skipped 1 previous similar message [ 2240.164898] Lustre: lustre-MDT0001: new disk, initializing [ 2240.284052] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2240.366627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2240.388910] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2246.177285] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2254.224688] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2258.725590] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2266.410618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2266.704752] Lustre: lustre-OST0000: new disk, initializing [ 2266.711864] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2266.723821] Lustre: 75469:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2266.812756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2274.123066] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2274.951805] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2274.968554] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2275.094414] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2283.133560] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2289.387560] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2289.884590] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2295.602372] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2295.859694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2300.896876] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2300.900966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2300.910270] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2300.919336] Lustre: Skipped 1 previous similar message [ 2300.937124] Lustre: Skipped 1 previous similar message [ 2306.027407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2306.037503] Lustre: Skipped 1 previous similar message [ 2312.159498] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2312.681954] Lustre: server umount lustre-OST0000 complete [ 2321.385580] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2321.400198] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2321.403113] Lustre: Skipped 1 previous similar message [ 2321.421459] Lustre: Skipped 3 previous similar messages [ 2323.771781] Lustre: server umount lustre-MDT0000 complete [ 2327.710251] LustreError: 75468:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781227134 with bad export cookie 1971340413687024790 [ 2327.729587] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2327.736330] LustreError: 75468:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2328.289515] Lustre: server umount lustre-MDT0001 complete [ 2337.382405] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 21:19:03 (1781227143) [ 2345.334391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2345.819501] LustreError: 77736:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2345.841409] LustreError: 77736:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2345.917110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2350.848681] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2351.074045] LustreError: 77737:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2355.171605] LustreError: 77736:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2360.305540] LustreError: 77737:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2360.522572] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2366.332143] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2372.867428] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2378.117880] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2385.709689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2385.901319] LustreError: 79331:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2386.118309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2386.121182] Lustre: Skipped 1 previous similar message [ 2393.219666] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2399.534559] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2403.929604] Lustre: 80068:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 60606a93-8b0d-47d3-93ff-4848e5cf9ce0 at adminstrative request [ 2405.863198] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2405.872994] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2416.116461] Lustre: MGS: Client 60606a93-8b0d-47d3-93ff-4848e5cf9ce0 (at 0@lo) reconnecting [ 2416.128977] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x1b5b9ba9612fd008 to 0x1b5b9ba9612fd2a8 [ 2416.159327] Lustre: MGC192.168.201.136@tcp: Connection restored to 0@lo (at 0@lo) [ 2441.227259] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2441.407057] Lustre: lustre-OST0001: new disk, initializing [ 2441.409615] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2441.414821] Lustre: 80461:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2447.523464] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2449.448835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2449.456058] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2449.514873] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2455.210331] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2459.619093] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2459.647956] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2459.648239] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2459.658101] Lustre: Skipped 1 previous similar message [ 2464.468379] Lustre: server umount lustre-OST0001 complete [ 2464.746441] LustreError: 79332:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2464.757133] LustreError: 79332:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2469.857753] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2469.881100] Lustre: Skipped 1 previous similar message [ 2474.873685] Lustre: server umount lustre-OST0000 complete [ 2479.075809] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2485.226211] LustreError: 77736:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2485.247257] LustreError: 77736:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2485.328311] Lustre: server umount lustre-MDT0000 complete [ 2489.563186] LustreError: 77719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781227296 with bad export cookie 1971340413687026344 [ 2489.577044] LustreError: 77719:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2489.578153] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2489.823624] Lustre: server umount lustre-MDT0001 complete [ 2498.331625] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 21:21:43 (1781227303) [ 2500.362715] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2502.449902] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 21:21:47 (1781227307) [ 2503.977742] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2505.864612] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 21:21:51 (1781227311) [ 2507.795403] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2509.645896] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 21:21:55 (1781227315) [ 2511.429048] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2513.721467] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 21:21:58 (1781227318) [ 2515.645925] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2522.155318] loop: module loaded [ 2522.175774] loop0: detected capacity change from 0 to 4294967296 [ 2522.248067] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2524.177496] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2528.257830] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2534.299751] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2570.179184] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2570.210968] LustreError: 84454:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2570.217857] LustreError: 84454:0:(obd_mount.c:250:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2570.225747] LustreError: 84454:0:(tgt_mount.c:2534:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2570.230839] LustreError: 84454:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2593.706475] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 2605.772866] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 2645.052895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2645.326368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2645.405932] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2645.542498] Lustre: lustre-MDT0000: new disk, initializing [ 2645.724747] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2645.732147] Lustre: Skipped 1 previous similar message [ 2645.750894] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2650.360653] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2661.802584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2661.906754] Lustre: 87289:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2662.248867] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2667.335860] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2677.235697] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2681.658830] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2690.077131] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2690.522127] Lustre: 89029:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2695.814690] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2695.953878] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2698.526617] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2707.003462] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2712.370510] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2712.572854] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2717.855439] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2718.143676] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2721.775516] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2721.776509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2721.796599] LustreError: Skipped 1 previous similar message [ 2721.813749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2721.820098] Lustre: Skipped 4 previous similar messages [ 2721.858094] Lustre: Skipped 6 previous similar messages [ 2726.446915] Lustre: server umount lustre-OST0000 complete [ 2737.110582] Lustre: server umount lustre-MDT0000 complete [ 2737.143579] LustreError: 87296:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2737.172262] LustreError: 87296:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2742.941095] LustreError: 89039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781227549 with bad export cookie 1971340413687026960 [ 2742.941820] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2742.966397] LustreError: 89039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2743.577945] Lustre: server umount lustre-MDT0001 complete [ 2753.995313] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 21:25:59 (1781227559) [ 2760.516801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2765.716385] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2775.708262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2776.110073] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2776.117114] Lustre: Skipped 3 previous similar messages [ 2781.581849] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2788.047414] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2792.155942] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2799.431285] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2806.474367] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2812.811664] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2818.140799] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2818.419853] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2823.586450] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2823.841841] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2832.845396] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2833.073653] Lustre: lustre-OST0001: new disk, initializing [ 2833.076894] Lustre: Skipped 2 previous similar messages [ 2833.080789] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2833.085922] Lustre: Skipped 3 previous similar messages [ 2833.091242] Lustre: 94294:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2839.166057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2839.177786] Lustre: Skipped 2 previous similar messages [ 2839.187582] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2839.270552] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2840.337988] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2849.316112] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2852.634456] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2852.822785] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2857.594471] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2857.827396] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2881.220126] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2908.738745] Lustre: Failing over lustre-MDT0000 [ 2909.157153] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2909.165379] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2909.184278] Lustre: Skipped 2 previous similar messages [ 2909.199889] LustreError: 91298:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2909.221168] LustreError: 91298:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [ 2909.528187] Lustre: server umount lustre-MDT0000 complete [ 2927.583384] Lustre: 64380:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781227718/real 1781227718] req@00000000088d2b93 x1867751632420352/t0(0) o400->MGC192.168.201.136@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1781227734 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2927.620797] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2937.395402] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2937.404302] LDISKFS-fs (dm-0): recovery complete [ 2937.416401] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2939.314427] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2943.226897] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2943.229877] Lustre: 91293:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2944.496158] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2944.501285] Lustre: 91299:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2944.526421] Lustre: 91299:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 1 previous similar message [ 2944.648448] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2944.751280] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 2944.977210] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2945.331855] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2945.349984] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2945.679738] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2945.700911] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 1 previous similar message [ 2945.721031] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2945.745094] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 7 previous similar messages [ 2946.413938] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2946.423564] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2946.743685] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2946.760764] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 2 previous similar messages [ 2947.776054] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2947.782652] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 5 previous similar messages [ 2948.440925] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2948.450779] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 2948.778708] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2948.783030] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 5 previous similar messages [ 2951.981681] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2952.007792] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 12 previous similar messages [ 2952.660695] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2952.678252] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2953.001315] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2953.011526] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 12 previous similar messages [ 2960.064054] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2960.087185] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 24 previous similar messages [ 2960.736520] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2960.740699] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 2961.055315] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2961.068480] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 24 previous similar messages [ 2976.340068] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2976.379205] Lustre: 96998:0:(ldlm_lib.c:2069:extend_recovery_timer()) Skipped 50 previous similar messages [ 2977.035636] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2977.054244] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 2977.383353] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2977.399116] LustreError: 96998:0:(ldlm_lib.c:2682:replay_request_or_update()) Skipped 50 previous similar messages [ 3003.255133] Lustre: 96998:0:(ldlm_lib.c:2378:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3003.263092] Lustre: 96998:0:(ldlm_lib.c:2388:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3003.274363] Lustre: 96998:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5836a909-b48e-41a2-b14d-8221ec756a24@192.168.201.36@tcp [ 3003.299849] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3003.307221] Lustre: 96998:0:(ldlm_lib.c:1898:abort_req_replay_queue()) @@@ aborted: req@000000005da38faf x1867751610194432/t0(8589934780) o35->5836a909-b48e-41a2-b14d-8221ec756a24@192.168.201.36@tcp:110/0 lens 392/0 e 0 to 0 dl 1781227820 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3003.348257] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3003.370615] Lustre: lustre-MDT0000: Denying connection for new client 5836a909-b48e-41a2-b14d-8221ec756a24 (at 192.168.201.36@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3003.390252] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3003.453672] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3003.461707] Lustre: Skipped 2 previous similar messages [ 3003.468114] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3003.518486] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3003.523483] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3021.312391] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3021.328923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3021.329959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3021.347547] Lustre: Skipped 4 previous similar messages [ 3021.366841] Lustre: Skipped 3 previous similar messages [ 3026.513870] Lustre: server umount lustre-OST0000 complete [ 3037.706869] Lustre: server umount lustre-OST0001 complete [ 3048.571154] Lustre: server umount lustre-MDT0000 complete [ 3052.023601] LustreError: 92211:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3052.054733] LustreError: 92211:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 53 previous similar messages [ 3052.352712] LustreError: 91280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781227859 with bad export cookie 1971340413687078179 [ 3052.354864] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3052.360232] LustreError: 91280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3052.762842] Lustre: server umount lustre-MDT0001 complete [ 3063.131943] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 21:31:08 (1781227868) [ 3071.621324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3071.637167] Lustre: *** cfs_fail_loc=197, val=0*** [ 3087.327939] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3092.448246] LustreError: 98956:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.136@tcp: failed processing log, type 4: rc = -110 [ 3097.678818] Lustre: *** cfs_fail_loc=197, val=0*** [ 3097.689836] Lustre: Skipped 65 previous similar messages [ 3118.175316] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3118.191524] Lustre: Skipped 3 previous similar messages [ 3126.254853] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3133.050795] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3135.566107] Lustre: server umount lustre-OST0000 complete [ 3144.576725] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 21:32:30 (1781227950) [ 3167.556242] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3173.249919] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3181.242670] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3185.306523] LustreError: 100495:0:(osp_object.c:617:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3185.320754] LustreError: 100495:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3185.335148] LustreError: 100495:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3185.652674] Lustre: server umount lustre-MDT0000 complete [ 3218.718877] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 3231.533567] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 3270.252168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3270.552726] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3270.579136] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3270.685977] Lustre: lustre-MDT0000: new disk, initializing [ 3270.792581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3277.216648] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3290.265222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3290.405163] Lustre: 104523:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3290.671575] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3295.203079] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3302.781688] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3306.692184] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3313.723660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3314.004713] Lustre: 106264:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3320.549906] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3320.660983] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3328.902134] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3333.611282] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3333.866537] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3338.396704] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3338.600187] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3341.282868] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3341.293197] Lustre: Skipped 4 previous similar messages [ 3341.294485] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3341.294756] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3341.294760] LustreError: Skipped 1 previous similar message [ 3341.341076] Lustre: Skipped 8 previous similar messages [ 3346.721652] Lustre: server umount lustre-OST0000 complete [ 3357.415504] Lustre: server umount lustre-MDT0000 complete [ 3361.519017] LustreError: 104513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228168 with bad export cookie 1971340413687088875 [ 3361.530497] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3361.987471] Lustre: server umount lustre-MDT0001 complete [ 3371.379309] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 21:36:16 (1781228176) [ 3382.297162] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 3385.075304] Key type lgssc unregistered [ 3385.548981] LNet: 108588:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3385.560978] LNetError: 108588:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3385.580469] LNet: Removed LNI 192.168.201.136@tcp [ 3386.837449] Key type .llcrypt unregistered [ 3386.839811] Key type ._llcrypt unregistered [ 3410.375319] Key type ._llcrypt registered [ 3410.379640] Key type .llcrypt registered [ 3410.670459] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 3430.648398] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 3432.068936] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3432.489258] alg: No test for adler32 (adler32-zlib) [ 3433.603893] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 3433.890909] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 3435.599239] Key type lgssc registered [ 3436.865867] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3478.105557] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3478.139095] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3479.479440] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3479.570311] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3479.671183] Lustre: lustre-MDT0000: new disk, initializing [ 3479.819064] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3479.864389] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3484.639272] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3496.436086] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3496.627707] Lustre: lustre-OST0000: new disk, initializing [ 3496.631516] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3496.635704] Lustre: Skipped 1 previous similar message [ 3496.640762] Lustre: 112716:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3496.729423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3505.247873] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3505.328461] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3505.346550] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3505.406562] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3517.246828] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3517.392935] Lustre: lustre-OST0001: new disk, initializing [ 3517.398323] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3517.406285] Lustre: 113562:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3517.513871] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3522.656467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3522.662448] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3522.687495] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3523.588972] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3537.889154] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3537.893598] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3537.908859] Lustre: Skipped 1 previous similar message [ 3541.285961] Lustre: server umount lustre-MDT0000 complete [ 3548.580163] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3553.710330] LustreError: 112715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228360 with bad export cookie 9339913329379834734 [ 3553.711446] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3553.719417] LustreError: 112715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3553.837908] Lustre: server umount lustre-OST0000 complete [ 3557.944270] Lustre: server umount lustre-OST0001 complete [ 3567.229435] Lustre: DEBUG MARKER: == conf-sanity test 88: check the default mount options can be overridden ========================================================== 21:39:32 (1781228372) [ 3578.311811] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 3603.091648] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 21:40:08 (1781228408) [ 3648.282786] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 3660.770688] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 3701.690938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3702.153556] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3702.194888] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3702.321462] Lustre: lustre-MDT0000: new disk, initializing [ 3702.507177] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3702.530906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3707.868876] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3720.220148] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3720.388746] Lustre: 120249:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3720.435501] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3720.438488] Lustre: Skipped 1 previous similar message [ 3720.502293] Lustre: lustre-MDT0001: new disk, initializing [ 3720.631231] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3720.676254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3720.692213] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3725.027921] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3733.968665] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3739.356974] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3748.697180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3748.972618] Lustre: lustre-OST0000: new disk, initializing [ 3748.975250] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3748.980793] Lustre: 121992:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3749.041273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3756.137443] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3756.153235] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3756.430880] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3757.530266] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3765.568208] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3770.102982] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3770.488406] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3775.651492] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3775.838446] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3782.112612] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3782.114981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3782.122050] LustreError: Skipped 1 previous similar message [ 3782.152115] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3782.168068] Lustre: Skipped 2 previous similar messages [ 3787.244317] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3787.257464] Lustre: Skipped 1 previous similar message [ 3792.352426] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3792.359495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3792.384614] Lustre: Skipped 1 previous similar message [ 3792.615889] Lustre: server umount lustre-OST0000 complete [ 3797.472737] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3797.477830] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3797.479928] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3797.486499] Lustre: Skipped 2 previous similar messages [ 3802.914709] Lustre: server umount lustre-MDT0000 complete [ 3806.667377] LustreError: 120241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228613 with bad export cookie 9339913329379835959 [ 3806.672169] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3806.676251] LustreError: 120241:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3807.014773] Lustre: server umount lustre-MDT0001 complete [ 3815.940798] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 21:43:41 (1781228621) [ 3822.814621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3823.316615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3827.790215] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3828.712389] LustreError: 124259:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3828.726869] LustreError: 124259:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 3833.826199] LustreError: 124260:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3836.276963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3840.771312] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3846.939193] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3851.142688] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3858.049794] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3858.541984] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3858.548498] Lustre: Skipped 1 previous similar message [ 3865.476408] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 3871.785335] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3876.864377] Lustre: *** cfs_fail_loc=159, val=0*** [ 3892.034895] Lustre: lustre-MDT0000: Client 3765bcaf-33ee-4246-b350-aac54c63005d (at 192.168.201.36@tcp) reconnecting [ 3893.363377] Lustre: *** cfs_fail_loc=159, val=0*** [ 3893.367934] Lustre: Skipped 5 previous similar messages [ 3909.930330] Lustre: lustre-MDT0000: Client 3765bcaf-33ee-4246-b350-aac54c63005d (at 192.168.201.36@tcp) reconnecting [ 3928.550837] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3928.559147] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3928.572157] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3928.578120] Lustre: Skipped 3 previous similar messages [ 3930.608286] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3932.952831] Lustre: server umount lustre-OST0000 complete [ 3939.296112] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3939.306236] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3939.315290] Lustre: Skipped 1 previous similar message [ 3942.748420] Lustre: server umount lustre-MDT0000 complete [ 3945.952973] LustreError: 124980:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3945.966889] LustreError: 124980:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 3946.460877] LustreError: 124239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781228753 with bad export cookie 9339913329379836841 [ 3946.463546] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3946.473240] LustreError: 124239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3946.852440] Lustre: server umount lustre-MDT0001 complete [ 3958.136574] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 3961.082124] Key type lgssc unregistered [ 3961.407622] LNet: 127689:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3961.412178] LNetError: 127689:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3961.433345] LNet: Removed LNI 192.168.201.136@tcp [ 3962.333477] Key type .llcrypt unregistered [ 3962.337466] Key type ._llcrypt unregistered [ 3986.659215] Key type ._llcrypt registered [ 3986.661932] Key type .llcrypt registered [ 3986.782613] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 3988.151879] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3988.178527] alg: No test for adler32 (adler32-zlib) [ 3989.254329] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 3989.470804] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 3991.167151] Key type lgssc registered [ 3992.226784] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3999.903533] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3999.930518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4001.703633] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4006.304665] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4012.009790] LustreError: 129088:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4012.050407] LustreError: 129088:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4014.876168] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4015.281392] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4019.447874] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4025.048545] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4028.328274] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4034.431154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4034.842936] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4040.843119] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4046.128797] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4047.294728] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4056.952882] Lustre: *** cfs_fail_loc=159, val=0*** [ 4073.288622] Lustre: lustre-MDT0000: Client f8f6dffc-490d-4170-a83e-42c19d6ebec1 (at 192.168.201.36@tcp) reconnecting [ 4076.158428] Lustre: *** cfs_fail_loc=159, val=0*** [ 4092.743939] Lustre: lustre-MDT0001: Client f8f6dffc-490d-4170-a83e-42c19d6ebec1 (at 192.168.201.36@tcp) reconnecting [ 4094.430934] Lustre: *** cfs_fail_loc=159, val=0*** [ 4094.438574] Lustre: Skipped 3 previous similar messages [ 4110.140412] Lustre: lustre-MDT0001: Client f8f6dffc-490d-4170-a83e-42c19d6ebec1 (at 192.168.201.36@tcp) reconnecting [ 4117.111891] Lustre: 131788:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4119.826105] Lustre: *** cfs_fail_loc=159, val=0*** [ 4119.830086] Lustre: Skipped 4 previous similar messages [ 4135.259530] Lustre: lustre-MDT0000: Client ec172df2-ba60-4183-86ba-673ac4bf9646 (at 192.168.201.36@tcp) reconnecting [ 4137.131952] Lustre: *** cfs_fail_loc=159, val=0*** [ 4137.147846] Lustre: Skipped 14 previous similar messages [ 4153.687319] Lustre: lustre-MDT0000: Client ec172df2-ba60-4183-86ba-673ac4bf9646 (at 192.168.201.36@tcp) reconnecting [ 4155.586655] Lustre: 132079:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4174.306958] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4174.322422] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4174.323025] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4174.327014] LustreError: Skipped 1 previous similar message [ 4174.335255] Lustre: Skipped 1 previous similar message [ 4179.457707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4179.463607] Lustre: Skipped 1 previous similar message [ 4179.833445] Lustre: server umount lustre-OST0000 complete [ 4184.545688] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4184.550571] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4184.551698] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4184.565838] Lustre: Skipped 2 previous similar messages [ 4189.671838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4189.678485] Lustre: Skipped 2 previous similar messages [ 4194.785930] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4194.791766] Lustre: Skipped 1 previous similar message [ 4198.367133] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4198.560354] Lustre: server umount lustre-MDT0000 complete [ 4199.911107] LustreError: 132027:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4199.933453] LustreError: 132027:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4202.540172] LustreError: 129807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229009 with bad export cookie 11639254035850879888 [ 4202.546867] LustreError: 129807:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4202.547670] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4202.954477] Lustre: server umount lustre-MDT0001 complete [ 4214.055631] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 4216.905298] Key type lgssc unregistered [ 4217.270338] LNet: 133144:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4217.278443] LNetError: 133144:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4217.307795] LNet: Removed LNI 192.168.201.136@tcp [ 4218.417121] Key type .llcrypt unregistered [ 4218.421069] Key type ._llcrypt unregistered [ 4244.833328] Key type ._llcrypt registered [ 4244.835302] Key type .llcrypt registered [ 4244.931749] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 4246.349116] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4246.387421] alg: No test for adler32 (adler32-zlib) [ 4247.333268] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 4247.492770] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 4249.167408] Key type lgssc registered [ 4250.914802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4259.629692] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4259.637679] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4261.007834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4265.236477] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4270.564421] LustreError: 134544:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4270.579303] LustreError: 134544:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4274.463839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4274.850236] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4280.425341] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4286.206308] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4291.398959] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4299.527400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4299.684239] LustreError: 136136:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4299.922796] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4305.402221] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4306.433144] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4306.803146] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4312.856625] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4319.814883] Lustre: 136783:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4322.642867] Lustre: 136832:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4323.857907] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4324.655831] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4328.854938] Lustre: 136980:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4329.848578] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4330.939878] Lustre: 137079:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4336.104868] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4336.120218] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4336.134838] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4337.126301] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4337.133439] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4337.133632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4341.831501] Lustre: server umount lustre-OST0000 complete [ 4346.338250] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4346.355408] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4346.369388] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4346.372241] Lustre: Skipped 1 previous similar message [ 4352.282698] LustreError: 137388:0:(ldlm_resource.c:1180:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x2:0x0].0x0 (000000005bd30025) refcount nonzero (1) after lock cleanup; forcing cleanup. [ 4352.386365] Lustre: server umount lustre-MDT0000 complete [ 4352.482719] LustreError: 134544:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4352.504574] LustreError: 134544:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4356.646812] LustreError: 134525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229163 with bad export cookie 14158533988190124790 [ 4356.651385] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4356.657741] LustreError: 134525:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4356.931923] Lustre: server umount lustre-MDT0001 complete [ 4369.106873] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 4372.453093] Key type lgssc unregistered [ 4372.855698] LNet: 138123:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4372.865728] LNetError: 138123:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4372.908198] LNet: Removed LNI 192.168.201.136@tcp [ 4374.011153] Key type .llcrypt unregistered [ 4374.016535] Key type ._llcrypt unregistered [ 4400.984325] Key type ._llcrypt registered [ 4400.987081] Key type .llcrypt registered [ 4401.179816] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 4402.668214] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4402.704651] alg: No test for adler32 (adler32-zlib) [ 4403.812624] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 4404.209387] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 4405.927255] Key type lgssc registered [ 4407.385766] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4415.883776] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4415.902473] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4417.686871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4422.637699] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4427.234260] LustreError: 139521:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4427.299130] LustreError: 139521:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4432.289142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4432.352804] LustreError: 139522:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4432.681876] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4437.739933] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4443.835053] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4448.169717] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4455.683223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4456.255543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4458.314354] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4463.071992] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4463.424589] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4468.594933] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4474.309954] Lustre: *** cfs_fail_loc=159, val=0*** [ 4474.319461] Lustre: Skipped 1 previous similar message [ 4490.016041] Lustre: lustre-MDT0000: Client 42f9ba15-5a03-4e7e-ad56-62e92d675750 (at 192.168.201.36@tcp) reconnecting [ 4509.671763] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4509.681476] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4509.681751] LustreError: Skipped 1 previous similar message [ 4509.684843] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4509.721709] Lustre: Skipped 1 previous similar message [ 4513.411573] Lustre: server umount lustre-OST0000 complete [ 4519.914977] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4519.921865] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4519.946738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4519.957850] Lustre: Skipped 2 previous similar messages [ 4525.027858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4525.037093] Lustre: Skipped 1 previous similar message [ 4529.121202] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4529.138442] Lustre: Skipped 1 previous similar message [ 4532.191128] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4532.479071] Lustre: server umount lustre-MDT0000 complete [ 4534.242972] LustreError: 140643:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4534.268377] LustreError: 140643:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4536.318248] LustreError: 139503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229343 with bad export cookie 5138841217308195341 [ 4536.338439] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4536.341153] LustreError: 139503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4536.623488] Lustre: server umount lustre-MDT0001 complete [ 4548.159380] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 4550.898086] Key type lgssc unregistered [ 4551.240525] LNet: 142832:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4551.252310] LNetError: 142832:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4551.284250] LNet: Removed LNI 192.168.201.136@tcp [ 4552.038586] Key type .llcrypt unregistered [ 4552.041443] Key type ._llcrypt unregistered [ 4577.522243] Key type ._llcrypt registered [ 4577.523687] Key type .llcrypt registered [ 4577.638375] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 4579.027985] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4579.043757] alg: No test for adler32 (adler32-zlib) [ 4580.110419] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 4580.358567] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 4582.087166] Key type lgssc registered [ 4583.516755] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4591.030731] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4591.038215] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4592.316925] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4597.862603] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4602.851784] LustreError: 144232:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4602.885196] LustreError: 144232:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4607.579824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4607.981261] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 4608.171663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4615.874935] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4623.854314] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4628.231683] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4636.075157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4636.517655] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4637.555544] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4642.032735] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4647.267913] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4652.916192] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4672.663785] Lustre: lustre-OST0000: evicting c9a953ae-5999-44aa-9fc6-756e979ce226 (at 192.168.201.36@tcp) by administrative request [ 4672.689902] Lustre: Skipped 1 previous similar message [ 4704.231920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4704.246244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4704.246667] LustreError: Skipped 1 previous similar message [ 4704.249946] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4704.287596] Lustre: Skipped 1 previous similar message [ 4707.579217] Lustre: server umount lustre-OST0000 complete [ 4714.468044] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4714.480349] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4714.480496] Lustre: Skipped 1 previous similar message [ 4714.495765] Lustre: Skipped 2 previous similar messages [ 4718.382316] Lustre: server umount lustre-MDT0000 complete [ 4719.599697] LustreError: 146803:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4719.614820] LustreError: 146803:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4723.018818] LustreError: 144212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229529 with bad export cookie 15350534072801397554 [ 4723.022319] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4723.039794] LustreError: 144212:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4723.377902] Lustre: server umount lustre-MDT0001 complete [ 4735.339150] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 4738.771549] Key type lgssc unregistered [ 4739.236632] LNet: 147948:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4739.252341] LNetError: 147948:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4740.330369] LNet: Removed LNI 192.168.201.136@tcp [ 4741.407484] Key type .llcrypt unregistered [ 4741.410977] Key type ._llcrypt unregistered [ 4816.983990] Key type ._llcrypt registered [ 4816.986673] Key type .llcrypt registered [ 4817.075505] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 4818.395345] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4818.580864] alg: No test for adler32 (adler32-zlib) [ 4819.631466] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 4819.845972] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 4821.528046] Key type lgssc registered [ 4822.876616] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4830.540284] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4830.562283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4832.253859] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4837.979443] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4842.472623] LustreError: 150423:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4842.516381] LustreError: 150423:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4847.594683] LustreError: 150424:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4849.608145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4850.433404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4856.839369] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4864.253283] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4868.207969] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4875.704498] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4876.129760] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4877.201144] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4882.863726] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 4889.053641] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4895.622225] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4902.884382] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4902.890193] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4902.904465] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4905.970188] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4905.974831] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4906.002073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4908.954959] Lustre: server umount lustre-OST0000 complete [ 4916.193254] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4916.196053] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4916.210745] Lustre: Skipped 1 previous similar message [ 4916.216783] Lustre: Skipped 1 previous similar message [ 4919.343474] Lustre: server umount lustre-MDT0000 complete [ 4921.318781] LustreError: 150420:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4921.335836] LustreError: 150420:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4923.593397] LustreError: 150403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229730 with bad export cookie 7971293791599925823 [ 4923.594128] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4923.613542] LustreError: 150403:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4924.105907] Lustre: server umount lustre-MDT0001 complete [ 4936.395568] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 4939.574622] Key type lgssc unregistered [ 4939.967760] LNet: 153641:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4940.001639] LNetError: 153641:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4941.029235] LNet: Removed LNI 192.168.201.136@tcp [ 4942.106256] Key type .llcrypt unregistered [ 4942.108294] Key type ._llcrypt unregistered [ 4970.775957] Key type ._llcrypt registered [ 4970.780210] Key type .llcrypt registered [ 4970.925248] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 4972.413553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4972.480343] alg: No test for adler32 (adler32-zlib) [ 4973.545977] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 4973.819710] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 4975.519445] Key type lgssc registered [ 4976.611586] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4993.003970] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 22:03:18 (1781229798) [ 5013.905387] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 5026.072517] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 5067.410064] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5067.456541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5068.790375] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5068.840435] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5068.910855] Lustre: lustre-MDT0000: new disk, initializing [ 5068.971180] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5068.988960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5073.718474] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5085.622216] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5085.730552] Lustre: 157861:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5085.808420] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5085.817450] Lustre: Skipped 1 previous similar message [ 5085.980196] Lustre: lustre-MDT0001: new disk, initializing [ 5086.122033] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5086.176731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5086.191296] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5090.985273] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5098.543486] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5102.802402] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5111.662173] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5112.065203] Lustre: lustre-OST0000: new disk, initializing [ 5112.069210] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5112.076758] Lustre: 159600:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5112.175262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5118.170418] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5121.674183] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5121.699297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5121.948421] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5126.802996] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5142.508747] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5142.509798] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5142.527265] LustreError: Skipped 1 previous similar message [ 5142.527924] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5142.570644] Lustre: Skipped 1 previous similar message [ 5144.553609] Lustre: server umount lustre-OST0000 complete [ 5152.740766] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5152.746604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5152.748738] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5152.748748] Lustre: Skipped 1 previous similar message [ 5152.784879] Lustre: Skipped 1 previous similar message [ 5155.543424] Lustre: server umount lustre-MDT0000 complete [ 5160.013503] LustreError: 157852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781229966 with bad export cookie 859213748492967512 [ 5160.027749] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5160.543709] Lustre: server umount lustre-MDT0001 complete [ 5173.510511] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 5176.294839] Key type lgssc unregistered [ 5176.620868] LNet: 161375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5176.631121] LNetError: 161375:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5176.649697] LNet: Removed LNI 192.168.201.136@tcp [ 5177.459786] Key type .llcrypt unregistered [ 5177.461586] Key type ._llcrypt unregistered [ 5204.931464] Key type ._llcrypt registered [ 5204.933541] Key type .llcrypt registered [ 5205.040891] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 5205.992707] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5206.039750] alg: No test for adler32 (adler32-zlib) [ 5207.235204] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 5207.413823] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 5209.071213] Key type lgssc registered [ 5210.376723] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5218.566965] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5218.586668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5220.174232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5224.394409] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5230.561488] LustreError: 162773:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5230.576850] LustreError: 162773:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5233.383599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5233.759267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5238.829647] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5244.169558] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5248.190207] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5255.124164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5255.493776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5262.296738] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 5265.787113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5269.029167] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5275.943879] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5275.975701] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5275.997608] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5276.011653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.028139] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5277.221574] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5277.252728] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5277.273404] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5277.287654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.304043] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5278.206193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.035870] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5279.039270] Lustre: Skipped 1 previous similar message [ 5279.052838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5279.064382] Lustre: Skipped 1 previous similar message [ 5279.069280] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5279.071474] Lustre: Skipped 1 previous similar message [ 5279.087102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.104385] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5279.111805] Lustre: Skipped 1 previous similar message [ 5280.246416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.208555] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5281.217237] Lustre: Skipped 1 previous similar message [ 5281.246594] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5281.258556] Lustre: Skipped 1 previous similar message [ 5281.278772] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5281.296301] Lustre: Skipped 1 previous similar message [ 5281.303515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.310548] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5281.321562] Lustre: Skipped 1 previous similar message [ 5282.412824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.447721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.452671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.657032] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5285.669275] Lustre: Skipped 3 previous similar messages [ 5285.688442] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5285.699577] Lustre: Skipped 3 previous similar messages [ 5285.708828] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5285.717493] Lustre: Skipped 3 previous similar messages [ 5285.728242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.740954] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5285.747784] Lustre: Skipped 3 previous similar messages [ 5286.869336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.692084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.685883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.688034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.865312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.742239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.860295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.860529] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5293.868569] Lustre: Skipped 7 previous similar messages [ 5293.889379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5293.900440] Lustre: Skipped 7 previous similar messages [ 5293.904267] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5293.909711] Lustre: Skipped 7 previous similar messages [ 5293.913985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.924457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5293.927378] Lustre: Skipped 7 previous similar messages [ 5295.014796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.086273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.116957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.288912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.299314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.467293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.508418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.664744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.725640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.778549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.876503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.170299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.248760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.437288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.575716] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5310.584881] Lustre: Skipped 14 previous similar messages [ 5310.619987] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5310.646499] Lustre: Skipped 14 previous similar messages [ 5310.664234] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5310.673575] Lustre: Skipped 14 previous similar messages [ 5310.681118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.712862] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5310.722775] Lustre: Skipped 14 previous similar messages [ 5311.713021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.952477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.951625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.257839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.167757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.158632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.171629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.385320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.511831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.500554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.562925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.690217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.711902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.543757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.534428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.655189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.721456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.852323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.777588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.741067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.864138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.014930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.130933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.193881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.262259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.308178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.312841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.325775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.354676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.216025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.351286] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5343.355885] Lustre: Skipped 30 previous similar messages [ 5343.375082] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5343.390390] Lustre: Skipped 30 previous similar messages [ 5343.394496] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5343.399957] Lustre: Skipped 30 previous similar messages [ 5343.404673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.414427] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5343.417832] Lustre: Skipped 30 previous similar messages [ 5344.309979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.403562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.236306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.148885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.194811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.063478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.198802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.815221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.995578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.033240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.980791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.044591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.013834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.809932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.928719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.639936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.640894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.672269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.625878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.590598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.835992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.080680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.071792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.244156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.306472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.514781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.671750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.931517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.914206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.823628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.728912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.733435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.918185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.721523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.689251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.919577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.104470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.063430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.001630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.364385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.650023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.711551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.400766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.473845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.752250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.147311] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5391.159598] LustreError: 162793:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5391.184030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.744729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.844202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.220145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.448637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.608697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.849256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.414336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.376527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.067061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.752265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.879084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.963382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.781360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.765704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.420898] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5407.423191] Lustre: Skipped 60 previous similar messages [ 5407.429529] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5407.433236] Lustre: Skipped 60 previous similar messages [ 5407.434983] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5407.436847] Lustre: Skipped 60 previous similar messages [ 5407.439397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.443817] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5407.447281] Lustre: Skipped 60 previous similar messages [ 5408.262831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.074391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.040915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.007996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.090855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.930204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.840715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.584648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.623945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.059666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.079598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.291757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.299761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.555697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.616893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.898803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.090684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.331218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.320064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.255770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.124858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.041617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.007848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.084406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.922325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.044936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.198992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.423608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.510575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.553592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.582679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.769854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.823838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.602616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.639146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.843689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.728216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.869494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.849662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.784883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.667544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.803704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.842081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.709267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.803165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.839562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.863250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.996174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.689611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.630546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.464672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.405551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.258531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.170530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.338315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.685905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.824320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.011147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.058213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.148078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.010534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.975343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.914255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.966599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.965448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.113234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.306253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.288462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.173265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.129630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.161971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.875695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.766669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.751750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.609729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.543847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.780827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.865553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.025400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.008585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.286770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.178513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.958239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.966943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.802943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.680167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.063045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.540630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.768263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.029099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.172953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.250871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.444946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.832160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.213820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.561301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.349700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.744683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.981896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.383274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.562584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.719800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.311531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.802968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.609603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.589365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.042396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.521137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.802832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.947413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.241969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.237969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.792848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.777839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.989594] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5533.994500] LustreError: 162793:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5534.004600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.003063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.971681] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5535.982114] Lustre: Skipped 116 previous similar messages [ 5536.017141] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5536.028866] Lustre: Skipped 116 previous similar messages [ 5536.031981] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5536.035087] Lustre: Skipped 116 previous similar messages [ 5536.037458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.047830] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5536.052160] Lustre: Skipped 116 previous similar messages [ 5537.293746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.225850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.278526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.138069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.063362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.922703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.880883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.750960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.937423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.112299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.064967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.974044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.035257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.172869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.378233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.347463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.368747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.163103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.058426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.036346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.011887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.820619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.712135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.696629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.673749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.525599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.725073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.681535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.788511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.746569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.716773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.593501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.700184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.588976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.567968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.538630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.491209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.422328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.336726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.203281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.166099] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5576.171829] LustreError: 162793:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5576.186240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.218227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.096530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.949464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.756904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.580351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.427628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.367957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.339973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.203413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.177429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.065862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.065101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.102105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.935363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.975333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.121300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.062583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.080264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.107531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.888908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.926251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.800800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.752528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.655231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.501159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.364415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.417330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.323396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.242139] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5603.255226] LustreError: 162793:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5603.256493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.124252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.988041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.804284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.736081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.655955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.407707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.430969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.323211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.259890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.415409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.535585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.605674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.539887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.529052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.527850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.506238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.392162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.403477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.433621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.427029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.281913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.248352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.993808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.797523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.646320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.434795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.266294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.504381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.410993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.282435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.063738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.963404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.738822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.614124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.147534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.276297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.529609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.937613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.085754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.380256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.512344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.237830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.152281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.312656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.696677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.646511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.176561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.533656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.413923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.704987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.152102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.965587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.040386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.951832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.857446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.889951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.150582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.295956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.449650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.447673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.409401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.165511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.251908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.134385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.069359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.880609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.942424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.912951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.059725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.904096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.103551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.128775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.413975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.572625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.819583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.806390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.047138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.122969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.322205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.278599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.418856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.543505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.475868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.653641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.533357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.555569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.548458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.658911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.502117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.468813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.437454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.498814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.487155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.504436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.700918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.676899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.770598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.654211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.528200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.437808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.407833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.354890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.378350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.230777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.203084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.223333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.180368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.122098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.179404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.109472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.156467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.311448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.198965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.427472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.463067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.676963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.761093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.597833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.538899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.394530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.430573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.510577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.559772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.492705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.646734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.695612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.788614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.999434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.016059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.965652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.958804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.864633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.641666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.543379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.355520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.249240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.122389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.869884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.001628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.098653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.170799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.165731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.964849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.876213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.018084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.846668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.832926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.681307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.914496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.033210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.100066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.004298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.021504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.032203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.078733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.017674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.019051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.923481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.048379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.076355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.976294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.013708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.110065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.862264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.727408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.355317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.132350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.202437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.317380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.104418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.327326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.329137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.268939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.103037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.423757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.525286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.859874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.140313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.456322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.480971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.525189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.547051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.436555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.449086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.340730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.258590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.408157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.349290] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5792.354885] Lustre: Skipped 257 previous similar messages [ 5792.364721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5792.374143] Lustre: Skipped 257 previous similar messages [ 5792.388240] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5792.397933] Lustre: Skipped 257 previous similar messages [ 5792.407438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.415525] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5792.421266] Lustre: Skipped 257 previous similar messages [ 5793.546612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.656493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.618016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.787369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.777531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.856884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.889682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.029149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.011938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.111551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.090763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.168550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.076810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.202593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.278822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.582471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.615088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.565811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.581722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.746874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.682602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.701246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.648168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.595740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.616224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.675255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.744322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.557886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.409882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.355598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.416471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.351985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.637252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.664361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.745666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.818278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.823257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.974467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.854866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.806150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.746176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.800723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.836364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.858113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.741866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.850492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.705650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.826748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.987646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.027086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.038744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.829160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.902270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.981215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.874412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.757944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.810042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.832425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.714481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.639579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.624620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.417896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.441458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.379321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.624167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.582084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.722483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.614497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.791763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.843360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.863136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.677433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.548941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.414491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.446348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.509407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.428028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.315269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.154554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.962878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.802982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.459469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.353971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.260920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.173872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.199134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.092426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.130453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.071993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.059826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.139963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.090264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.231557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.208205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.153308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.963389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.836639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.678574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.605370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.254557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.174555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.010767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.941592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.898596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.972043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.690131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.739730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.667611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.798509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.727656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.660932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.428666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.444474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.189104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.140505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.092839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.101716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.889710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.030617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.975227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.003679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.873447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.848755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.741221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.862352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.982885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.040608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.018705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.901517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.883211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.783553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.561889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.391469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.147266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.080099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.993663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.143910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.094091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.139379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.004483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.082096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.056958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.014406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.052273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.169447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.128863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.255379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.167045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.222430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.129256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.237180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.237387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.263247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.420491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.357401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.471619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.395481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.489799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.346975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.402551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.272784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.127773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.111893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.237310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.117709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.841325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.708753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.517796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.410473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.634770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.760098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.966819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.176595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.165189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.073314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.282962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.475357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.400393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.261572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.153720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.439062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.447901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.608859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.521996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.493588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.471054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.421440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.097185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.088461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.918948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.835211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.890469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.077341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.296516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.311357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.189393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.295922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.247497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.276513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.230089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.120723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.199564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.006870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.997916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.015039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.144926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.138680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.046654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.199535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.098641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.128952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.963593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.955808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.023587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.933142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.955448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.811830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.772643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.720612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.811373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.827455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.780972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.858800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.835075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.950671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.830743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.525314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.369836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.359499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.335591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.287141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.947632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.772079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.579324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.507715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.475871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.527743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.354942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.230620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.108794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.044130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.842861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.812532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.665288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.438178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.327633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.319746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.277446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.563342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.446459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.510764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.366766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.473331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.635458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.687235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.782476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.914559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.240749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.422418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.743841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.865110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.032402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.245650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.396514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.654026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.827694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.915296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.197830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.239442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.309251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.497276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.686120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.698730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.787066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.016064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.985938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.268838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.305397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.275601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.400275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.528709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.716135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.894364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.983390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.273073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.397456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.588174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.397680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.001930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.373259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.709031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.113199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.374881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.431200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.554542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.658691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.687786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.602515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.505839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.615080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.679722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.754247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.106691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.556855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.830940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.358502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.760759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.018608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.501171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.723223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.219972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.890128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.391907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.765193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.807217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.499554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.154736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.281767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.577895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.027877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.299853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.261805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.127789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.841874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.692644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.773070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.761545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.826892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.773646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.274409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.549562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.416444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.649186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.709549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.800458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.585903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.615928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.571158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.810725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.508564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.680357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.594344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.638322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.562325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.346207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.217546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.183881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.048360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.939567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.809812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.984921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.002286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.035065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.797711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.849798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.105030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.347333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.397487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.332131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.402939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.418120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.455944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.558531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.614327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.588079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.570833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.593168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.578529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.646700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.664409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.835698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.107311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.272170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.412437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.543887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.758958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.749919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.045079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.081834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.158655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.344525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.174430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.290808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.229920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.177817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.937039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.904361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.971978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.026214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.958453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.090654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.057751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.166350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.995545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.184863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.125730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.473096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.597668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.760639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.989943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.834898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.923596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.843716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.635768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.537621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.500204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.388661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.455129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.547272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.578778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.615214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.650697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.556707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.638802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.641078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.505934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.464714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.493969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.595672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.693356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.709865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.774840] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6225.783078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.796569] LustreError: 162793:0:(osp_precreate.c:1356:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6226.611609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.548051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.251221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.149386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.028983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.942801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.908935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.911767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.688259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.608223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.417777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.379854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.399374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.423957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.491991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.421830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.389444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.360446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.628630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.659430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.692456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.746219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.719287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.675768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.508992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.633905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.664108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.580567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.530264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.513854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.482633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.280327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.317642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.107585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.156341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.169307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.041530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.006500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.255360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.484564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.436950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.507152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.524568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.513861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.338237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.327140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.234613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.138490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.069067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.202831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.198668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.246118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.068483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.245632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.216137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.150558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.324598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.333601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.527191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.542333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.547071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.393754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.577803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.779936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.937826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.275350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.208918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.394442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.677111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.969051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.015551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.211448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.104228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.996424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.165710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.544341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.640682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.682414] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6304.692171] Lustre: Skipped 499 previous similar messages [ 6304.732632] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6304.745942] Lustre: Skipped 499 previous similar messages [ 6304.753125] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6304.761884] Lustre: Skipped 499 previous similar messages [ 6304.767367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.792591] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6304.797695] Lustre: Skipped 499 previous similar messages [ 6677.992681] LustreError: 164369:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6677.999886] LustreError: 162793:0:(osp_precreate.c:652:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7117.280987] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7117.305066] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7117.322585] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7118.818081] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7118.831212] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7122.869737] Lustre: server umount lustre-OST0000 complete [ 7128.046099] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7128.050382] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7128.053030] Lustre: Skipped 2 previous similar messages [ 7134.689154] LustreError: 162774:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7134.701334] LustreError: 162774:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7135.682937] Lustre: server umount lustre-MDT0000 complete [ 7139.174299] LustreError: 162755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781231946 with bad export cookie 9716443576349628300 [ 7139.175539] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7139.187243] LustreError: 162755:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7139.456249] Lustre: server umount lustre-MDT0001 complete [ 7153.159607] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 7157.165913] Key type lgssc unregistered [ 7157.384066] LNet: 217649:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7157.390753] LNetError: 217649:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7157.412798] LNet: Removed LNI 192.168.201.136@tcp [ 7158.213230] Key type .llcrypt unregistered [ 7158.214658] Key type ._llcrypt unregistered [ 7186.062921] Key type ._llcrypt registered [ 7186.068551] Key type .llcrypt registered [ 7186.260594] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 7188.446994] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7188.580314] alg: No test for adler32 (adler32-zlib) [ 7190.261937] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 7190.663902] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 7192.351324] Key type lgssc registered [ 7193.908480] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7204.791746] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7204.849492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7206.648197] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7212.954373] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7216.103752] LustreError: 219049:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7216.131108] LustreError: 219049:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7221.225850] LustreError: 219050:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7222.892115] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7223.456948] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7228.645825] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7235.480569] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7239.553147] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7246.414244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7246.768262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7251.914521] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7252.380689] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7259.639702] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7265.498986] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7267.506424] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7267.521693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7267.526564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7267.539631] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7267.544169] LustreError: 219069:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7272.417393] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7272.459052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7272.462313] Lustre: Skipped 1 previous similar message [ 7277.540381] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7277.544778] Lustre: Skipped 2 previous similar messages [ 7282.657509] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7282.671433] Lustre: Skipped 2 previous similar messages [ 7284.191733] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7284.293788] Lustre: server umount lustre-OST0000 complete [ 7287.561244] LustreError: 219069:0:(osp_precreate.c:1264:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7318.050767] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7318.518300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7327.205887] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7334.143122] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7337.975040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.987617] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.002137] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7338.021086] Lustre: Skipped 1 previous similar message [ 7340.462179] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 4 sec [ 7345.298058] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a160304b800.ost_server_uuid 50 [ 7347.858564] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a160304b800.ost_server_uuid in FULL state after 0 sec [ 7353.314936] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7353.320731] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7353.331764] LustreError: Skipped 1 previous similar message [ 7353.334496] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7353.334506] Lustre: Skipped 1 previous similar message [ 7353.403859] Lustre: Skipped 2 previous similar messages [ 7355.024976] Lustre: server umount lustre-OST0000 complete [ 7361.507617] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7361.514862] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7361.545243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7361.554292] Lustre: Skipped 1 previous similar message [ 7366.073027] Lustre: server umount lustre-MDT0000 complete [ 7368.683703] LustreError: 220272:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7368.701678] LustreError: 220272:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7370.331670] LustreError: 219030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232177 with bad export cookie 17228945270986240053 [ 7370.335133] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7370.337958] LustreError: 219030:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7370.791497] Lustre: server umount lustre-MDT0001 complete [ 7384.636083] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 7388.567208] Key type lgssc unregistered [ 7389.002957] LNet: 223691:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7389.012523] LNetError: 223691:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7389.043243] LNet: Removed LNI 192.168.201.136@tcp [ 7390.466963] Key type .llcrypt unregistered [ 7390.468703] Key type ._llcrypt unregistered [ 7425.657953] Key type ._llcrypt registered [ 7425.660835] Key type .llcrypt registered [ 7425.740276] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 7427.486150] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7427.538682] alg: No test for adler32 (adler32-zlib) [ 7428.765663] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 7429.111318] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 7430.807547] Key type lgssc registered [ 7432.236713] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7442.564648] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7442.579292] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7444.118687] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7448.413815] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7453.666435] LustreError: 225410:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7453.692648] LustreError: 225410:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7458.417037] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7458.889371] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7463.445504] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7466.204476] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7472.654668] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7472.941898] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7480.995858] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7488.336676] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7490.929687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7491.135784] Lustre: lustre-OST0001: new disk, initializing [ 7491.139515] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7491.146902] Lustre: 227666:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7491.277170] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7497.823359] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7499.921018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7499.933267] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7500.126385] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7510.400943] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7519.303638] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7535.591380] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7535.604335] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7535.626836] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7536.134918] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7536.150774] Lustre: Skipped 1 previous similar message [ 7536.156761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7536.164979] Lustre: Skipped 3 previous similar messages [ 7538.718915] Lustre: server umount lustre-MDT0000 complete [ 7541.226943] LustreError: 225406:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7541.264661] LustreError: 225406:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 7543.104168] LustreError: 225391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232350 with bad export cookie 11797886417530361130 [ 7543.111269] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7543.120357] LustreError: 225391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7543.405596] Lustre: server umount lustre-MDT0001 complete [ 7558.221520] Lustre: server umount lustre-OST0000 complete [ 7562.207856] Lustre: 224672:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781232353/real 1781232353] req@ffff9cae8159e940 x1867757373192704/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781232369 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7562.263705] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7562.646899] Lustre: server umount lustre-OST0001 complete [ 7584.035902] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 7594.582871] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7595.157343] LustreError: 230741:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7595.227528] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7600.102802] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7600.617298] LustreError: 230742:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7605.729657] LustreError: 230741:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7607.878144] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7608.244291] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7613.046921] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7616.121700] Lustre: 231850:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7622.663406] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7630.062160] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7633.271134] LustreError: 232203:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7633.310989] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 7639.087721] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7639.272970] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7639.279154] Lustre: Skipped 1 previous similar message [ 7640.328587] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7640.332127] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 7646.239124] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7654.920898] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7658.958114] Lustre: 233685:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7671.265256] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7671.289324] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7671.303851] Lustre: Skipped 2 previous similar messages [ 7671.319501] Lustre: Skipped 3 previous similar messages [ 7676.385383] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7676.406515] Lustre: Skipped 3 previous similar messages [ 7681.511366] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7681.532887] Lustre: Skipped 3 previous similar messages [ 7684.063390] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7684.369902] Lustre: server umount mylustre-MDT0000 complete [ 7686.627543] LustreError: 230742:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7686.645838] LustreError: 230742:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7688.858590] LustreError: 233886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232495 with bad export cookie 11797886417530364000 [ 7688.868481] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7688.871482] LustreError: 233886:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7689.240402] Lustre: server umount mylustre-MDT0001 complete [ 7703.751136] Lustre: server umount mylustre-OST0000 complete [ 7716.733441] Lustre: server umount mylustre-OST0001 complete [ 7741.903599] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 7753.256098] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7753.714172] LustreError: 235982:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7753.805412] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7758.677703] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7767.350261] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7771.578471] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7774.249989] Lustre: 237091:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7780.850410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7787.611322] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7792.589268] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7796.820223] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7798.147474] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7798.160052] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:65) [ 7803.682695] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7812.577388] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7816.460131] Lustre: 238928:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7823.840184] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7823.845185] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7823.846452] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7823.883920] Lustre: Skipped 3 previous similar messages [ 7830.037485] Lustre: server umount tfs-MDT0000 complete [ 7834.074624] LustreError: 238930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232640 with bad export cookie 11797886417530366177 [ 7834.088059] Lustre: tfs-MDT0001-lwp-OST0000: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7834.099433] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7834.105751] LustreError: 238930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7834.122110] LustreError: 238799:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7834.129338] Lustre: Skipped 1 previous similar message [ 7834.190885] LustreError: 238799:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 7834.690230] Lustre: server umount tfs-MDT0001 complete [ 7841.353924] Lustre: server umount tfs-OST0000 complete [ 7846.150575] Lustre: server umount tfs-OST0001 complete [ 7872.290943] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 7884.562110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7885.073095] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7885.085429] Lustre: Skipped 3 previous similar messages [ 7890.495290] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7900.995105] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7907.220994] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7910.774904] Lustre: 242268:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7918.243495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7926.569678] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7926.933382] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7930.057279] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7938.408943] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7944.211503] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:97) [ 7945.282720] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 7952.798185] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7956.874690] Lustre: 244105:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7964.863200] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 22:52:50 (1781232770) [ 7983.585766] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7983.594852] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7983.615216] Lustre: Skipped 1 previous similar message [ 7983.627105] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7983.632252] Lustre: Skipped 9 previous similar messages [ 7995.364966] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7995.734263] Lustre: server umount lustre-MDT0000 complete [ 8000.198111] LustreError: 241889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781232807 with bad export cookie 11797886417530368095 [ 8000.212242] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8000.219657] LustreError: 241889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8000.480203] LustreError: 242318:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8000.497644] LustreError: 242318:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 11 previous similar messages [ 8006.962190] Lustre: server umount lustre-MDT0001 complete [ 8018.223538] Lustre: server umount lustre-OST0000 complete [ 8028.494780] Lustre: server umount lustre-OST0001 complete [ 8036.171989] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 8046.547466] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8083.199879] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8083.199879] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8083.199879] [ 8083.253527] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8083.485284] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8083.512590] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8083.577455] Lustre: lustre-MDT0000: new disk, initializing [ 8083.659710] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8083.666397] Lustre: Skipped 3 previous similar messages [ 8083.680695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8089.081301] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8099.414225] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8099.414225] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8099.414225] [ 8099.448153] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8099.504779] Lustre: 247258:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8099.542663] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8099.548485] Lustre: Skipped 1 previous similar message [ 8099.590368] Lustre: lustre-MDT0001: new disk, initializing [ 8099.694622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8099.715287] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8103.993882] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8115.853342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8116.021209] Lustre: lustre-OST0000: new disk, initializing [ 8116.023438] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8116.026485] Lustre: 248804:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8118.021161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8118.040742] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8118.249297] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8124.440269] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8135.040971] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8135.266932] Lustre: lustre-OST0001: new disk, initializing [ 8135.271173] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8135.277769] Lustre: 249657:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8137.136383] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8137.145605] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8137.331030] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8142.826607] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8155.637295] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 22:56:01 (1781232961) [ 8173.764526] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 22:56:19 (1781232979) [ 8193.534511] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8193.549632] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8193.578731] Lustre: Skipped 6 previous similar messages [ 8193.587540] Lustre: Skipped 17 previous similar messages [ 8196.513276] Lustre: server umount lustre-OST0000 complete [ 8202.218248] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8207.837422] Lustre: server umount lustre-MDT0000 complete [ 8213.062364] LustreError: 247248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233019 with bad export cookie 11797886417530369873 [ 8213.068894] LustreError: 247248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8213.071148] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8219.641510] Lustre: server umount lustre-MDT0001 complete [ 8231.052879] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 8242.652695] Lustre: server umount lustre-OST0001 complete [ 8249.314716] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 8266.232164] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8303.995345] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8304.247832] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8304.284713] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8304.372893] Lustre: lustre-MDT0000: new disk, initializing [ 8304.474907] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8309.476579] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8321.543917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8321.778296] Lustre: 254150:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8321.861443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8321.869459] Lustre: Skipped 1 previous similar message [ 8322.020084] Lustre: lustre-MDT0001: new disk, initializing [ 8322.253416] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8322.272843] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8326.487948] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8334.562380] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8338.573632] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8346.742903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8347.150249] Lustre: 255893:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8347.285522] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8347.297470] Lustre: Skipped 5 previous similar messages [ 8349.102325] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8349.233409] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8354.269469] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8362.437133] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8369.640994] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8369.641269] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8369.651573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8369.672692] Lustre: Skipped 5 previous similar messages [ 8369.692461] Lustre: Skipped 7 previous similar messages [ 8375.849965] Lustre: server umount lustre-OST0000 complete [ 8386.025372] Lustre: server umount lustre-MDT0000 complete [ 8389.635178] LustreError: 254143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233196 with bad export cookie 11797886417530371343 [ 8389.643332] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8389.645142] LustreError: 254143:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8390.125562] Lustre: server umount lustre-MDT0001 complete [ 8403.953383] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 8407.892465] Key type lgssc unregistered [ 8408.310474] LNet: 257527:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8408.327561] LNetError: 257527:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8408.340775] LNet: Removed LNI 192.168.201.136@tcp [ 8409.377136] Key type .llcrypt unregistered [ 8409.380392] Key type ._llcrypt unregistered [ 8442.908712] Key type ._llcrypt registered [ 8442.915750] Key type .llcrypt registered [ 8443.106212] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8444.717493] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8444.734972] alg: No test for adler32 (adler32-zlib) [ 8445.766311] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 8446.020338] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 8447.735656] Key type lgssc registered [ 8449.270197] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8458.643341] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8458.664889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8460.165809] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8465.405366] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8470.502239] LustreError: 258986:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8470.522251] LustreError: 258986:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8474.838190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8475.207787] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8479.484672] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8485.232971] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8489.047965] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8495.103808] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8495.421328] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8501.235855] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8505.796589] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8506.207738] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8507.299246] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8510.945423] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8510.964119] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8510.969560] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8510.981551] Lustre: Skipped 1 previous similar message [ 8511.001611] Lustre: Skipped 1 previous similar message [ 8515.266612] Lustre: server umount lustre-OST0000 complete [ 8521.188239] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8521.189887] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8521.195331] LustreError: Skipped 1 previous similar message [ 8521.202413] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8521.204249] Lustre: Skipped 1 previous similar message [ 8521.213840] Lustre: Skipped 1 previous similar message [ 8525.770822] Lustre: server umount lustre-MDT0000 complete [ 8526.322822] LustreError: 258982:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8526.363455] LustreError: 258982:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8529.792118] LustreError: 258967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233336 with bad export cookie 2943196681623369857 [ 8529.818525] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8530.147284] Lustre: server umount lustre-MDT0001 complete [ 8543.059231] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 8545.701340] Key type lgssc unregistered [ 8546.095285] LNet: 262216:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8546.100440] LNetError: 262216:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8546.122960] LNet: Removed LNI 192.168.201.136@tcp [ 8547.021845] Key type .llcrypt unregistered [ 8547.024326] Key type ._llcrypt unregistered [ 8564.450417] Key type ._llcrypt registered [ 8564.455395] Key type .llcrypt registered [ 8564.553518] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8565.168463] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8565.216715] alg: No test for adler32 (adler32-zlib) [ 8566.640459] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 8566.860057] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 8568.639315] Key type lgssc registered [ 8570.011601] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8579.336965] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8579.356495] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8594.847487] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8599.967201] LustreError: 263391:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.136@tcp: failed processing log, type 4: rc = -110 [ 8626.719779] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8633.267040] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8641.749208] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8644.676555] Lustre: server umount lustre-OST0000 complete [ 8661.833585] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 8664.197749] Key type lgssc unregistered [ 8664.490800] LNet: 264893:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8664.498876] LNetError: 264893:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8664.513719] LNet: Removed LNI 192.168.201.136@tcp [ 8665.184321] Key type .llcrypt unregistered [ 8665.185655] Key type ._llcrypt unregistered [ 8704.995909] Key type ._llcrypt registered [ 8704.999235] Key type .llcrypt registered [ 8705.120321] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8706.353033] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8706.554417] alg: No test for adler32 (adler32-zlib) [ 8707.917545] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 8708.310042] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 8710.120505] Key type lgssc registered [ 8711.397808] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8726.586312] loop0: detected capacity change from 0 to 204800000 [ 8726.840483] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8726.853680] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8727.349686] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8728.547323] loop0: detected capacity change from 0 to 204800000 [ 8728.779611] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8728.804309] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8729.404418] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8730.621631] loop0: detected capacity change from 0 to 204800000 [ 8730.903277] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8730.948125] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8731.079221] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8732.318315] loop0: detected capacity change from 0 to 204800000 [ 8732.514837] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8732.533021] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8732.675357] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8734.009293] loop0: detected capacity change from 0 to 204800000 [ 8734.050938] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8738.606060] loop0: detected capacity change from 0 to 204800000 [ 8738.652100] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8743.098303] loop0: detected capacity change from 0 to 204800000 [ 8743.133969] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8747.702131] loop0: detected capacity change from 0 to 204800000 [ 8747.749162] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8751.935216] loop0: detected capacity change from 0 to 204800000 [ 8752.032226] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8752.075544] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8752.146658] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8752.163711] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8752.171923] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8752.186343] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8752.199806] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8752.209143] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8752.216078] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8752.226714] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8752.241955] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8752.251782] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8752.264402] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8752.276028] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8752.281147] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8752.294225] Lustre: 268304:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8757.961619] Lustre: server umount MGS complete [ 8758.718806] loop0: detected capacity change from 0 to 204800000 [ 8758.842384] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8759.499769] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8759.558124] Lustre: lustre-MDT0000: Aborting client recovery [ 8760.451281] loop1: detected capacity change from 0 to 204800000 [ 8760.634709] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8760.688130] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8760.704074] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8760.713356] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8760.722606] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8760.730616] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8760.738269] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8760.744269] Lustre: 268737:0:(osd_scrub.c:1830:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8760.757384] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8760.907424] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8760.952373] Lustre: lustre-MDT0001: Aborting client recovery [ 8761.894563] loop2: detected capacity change from 0 to 204800000 [ 8762.081228] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8762.149082] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8762.154980] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8762.161907] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8762.171795] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8762.185372] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8762.202845] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8762.219648] Lustre: 268815:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8762.365491] Lustre: Found index 0 for lustre-OST0000, updating log [ 8762.418574] Lustre: 268833:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8762.479592] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8762.510911] Lustre: lustre-OST0000: Aborting recovery [ 8763.437597] loop3: detected capacity change from 0 to 204800000 [ 8763.667949] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8763.811731] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8763.825224] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8763.838606] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8763.853967] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8763.865855] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8763.873641] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8763.886657] Lustre: 268899:0:(osd_scrub.c:1830:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8763.905895] Lustre: Found index 1 for lustre-OST0001, updating log [ 8763.942771] Lustre: 268906:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8764.045944] Lustre: lustre-OST0001: Aborting recovery [ 8769.196788] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8769.199135] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8769.213150] LustreError: 268681:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8769.228694] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8769.233879] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8770.232879] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8770.237313] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8771.980772] LustreError: 268647:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1648us: rc = -2 [ 8772.485994] LustreError: 268643:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 848us: rc = -2 [ 8772.497352] LustreError: 268643:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 33 previous similar messages [ 8785.891377] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8785.893444] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8785.903175] Lustre: Skipped 3 previous similar messages [ 8785.922597] Lustre: Skipped 3 previous similar messages [ 8791.009926] LustreError: 268649:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8791.029848] LustreError: 268649:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8791.203166] Lustre: server umount lustre-MDT0000 complete [ 8792.152035] LustreError: 268905:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233599 with bad export cookie 7041428052888470963 [ 8792.158939] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8792.162865] LustreError: 268905:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8796.131862] LustreError: 268646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8796.138402] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8796.140650] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8796.149676] LustreError: 268646:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 8796.185820] Lustre: Skipped 1 previous similar message [ 8797.152596] LustreError: 268643:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8797.152843] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8797.166434] LustreError: 268643:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 8797.182126] Lustre: Skipped 2 previous similar messages [ 8798.583231] Lustre: server umount lustre-MDT0001 complete [ 8805.776458] Lustre: server umount lustre-OST0000 complete [ 8821.215172] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8821.375540] Lustre: server umount lustre-OST0001 complete [ 8829.278299] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 23:07:14 (1781233634) [ 8857.158595] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 8869.893033] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 8907.246104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8907.655327] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8907.700988] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8907.782367] Lustre: lustre-MDT0000: new disk, initializing [ 8907.935771] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8907.948869] Lustre: Skipped 1 previous similar message [ 8907.972724] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8912.830481] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8922.937267] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8923.080253] Lustre: 272938:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8923.147825] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8923.154778] Lustre: Skipped 1 previous similar message [ 8923.256800] Lustre: lustre-MDT0001: new disk, initializing [ 8923.352462] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8923.388733] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8923.399229] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8928.073205] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8935.860165] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 8939.314474] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 8945.587531] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8945.967779] Lustre: lustre-OST0000: new disk, initializing [ 8945.972701] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8945.977900] Lustre: 274682:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8946.087821] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8950.737976] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 8953.986604] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8953.995612] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8954.120285] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8958.400874] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 8965.842653] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8979.057743] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8987.209860] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9041.378442] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9041.382073] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9041.416205] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9041.429722] Lustre: Skipped 1 previous similar message [ 9043.591706] Lustre: server umount lustre-OST0000 complete [ 9051.618126] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9051.618842] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9051.635879] Lustre: Skipped 1 previous similar message [ 9051.647777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9051.674747] Lustre: Skipped 1 previous similar message [ 9053.595149] Lustre: server umount lustre-MDT0000 complete [ 9056.744095] LustreError: 272945:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9056.758441] LustreError: 272945:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9058.371298] LustreError: 272930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233865 with bad export cookie 7041428052888473679 [ 9058.374098] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9058.386028] LustreError: 272930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9058.799788] Lustre: server umount lustre-MDT0001 complete [ 9067.843949] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9072.553931] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9077.287358] Lustre: server umount MGS complete [ 9087.736532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9088.218128] LustreError: 278467:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9088.244120] LustreError: 278480:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9088.342704] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9092.426448] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9098.720978] LustreError: 278468:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9098.736038] LustreError: 278468:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9100.676142] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9105.559681] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9110.656173] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9115.033136] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9124.745731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9125.388099] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9125.391024] Lustre: Skipped 1 previous similar message [ 9135.254844] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9141.575105] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9172.457501] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9172.468936] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9172.485591] Lustre: Skipped 1 previous similar message [ 9172.491537] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9174.502771] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9176.714891] Lustre: server umount lustre-OST0000 complete [ 9183.203651] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9183.220270] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9183.232592] Lustre: Skipped 1 previous similar message [ 9187.461973] Lustre: server umount lustre-MDT0000 complete [ 9189.864322] LustreError: 278467:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9189.883500] LustreError: 278467:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9191.625143] LustreError: 280853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781233998 with bad export cookie 7041428052888475520 [ 9191.628914] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9191.644344] LustreError: 280853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9192.128411] Lustre: server umount lustre-MDT0001 complete [ 9204.679558] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 9207.895182] Key type lgssc unregistered [ 9208.197700] LNet: 282055:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9208.210732] LNetError: 282055:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9208.244322] LNet: Removed LNI 192.168.201.136@tcp [ 9209.421544] Key type .llcrypt unregistered [ 9209.423767] Key type ._llcrypt unregistered [ 9253.392949] Key type ._llcrypt registered [ 9253.395974] Key type .llcrypt registered [ 9253.538684] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 9272.586325] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 9273.862076] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9273.940241] alg: No test for adler32 (adler32-zlib) [ 9275.483676] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 9275.817886] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 9277.567186] Key type lgssc registered [ 9278.850278] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9314.484212] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9314.519599] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9315.945793] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9315.990857] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9316.094689] Lustre: lustre-MDT0000: new disk, initializing [ 9316.196650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9316.216544] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9321.458879] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9332.388938] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9332.536894] Lustre: 285982:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9332.597262] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9332.602526] Lustre: Skipped 1 previous similar message [ 9332.699660] Lustre: lustre-MDT0001: new disk, initializing [ 9332.814849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9332.896463] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9332.922064] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9337.930038] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9346.266326] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9350.532355] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9356.683151] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9356.917269] Lustre: lustre-OST0000: new disk, initializing [ 9356.920570] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9356.925206] Lustre: 287725:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9357.027416] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9363.512721] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9364.577793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9364.594217] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9364.798930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9371.668282] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9378.517784] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9391.981281] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9398.500475] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9441.761938] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9441.769663] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9441.785402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9441.802617] Lustre: Skipped 1 previous similar message [ 9446.898625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9446.914880] Lustre: Skipped 1 previous similar message [ 9447.741503] Lustre: server umount lustre-OST0000 complete [ 9452.004349] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9452.013737] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9452.023128] Lustre: Skipped 2 previous similar messages [ 9455.009394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9455.022851] Lustre: Skipped 2 previous similar messages [ 9458.046378] Lustre: server umount lustre-MDT0000 complete [ 9461.522533] LustreError: 285974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234268 with bad export cookie 11184384174204655945 [ 9461.528942] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9461.532297] LustreError: 285974:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9461.839795] Lustre: server umount lustre-MDT0001 complete [ 9470.280518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9474.834058] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9479.458571] Lustre: server umount MGS complete [ 9492.300967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9493.120327] LustreError: 291521:0:(qmt_pool.c:1630:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9493.237653] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9498.595533] LustreError: 291509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9498.623036] LustreError: 291509:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9499.303727] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9503.714290] LustreError: 291508:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9508.835631] LustreError: 291509:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9509.480669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9510.138847] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9516.176440] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9521.975137] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9526.252348] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9532.761714] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9533.229230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9541.187622] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9548.840765] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9580.522405] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9580.533650] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9580.549414] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9580.556252] Lustre: Skipped 2 previous similar messages [ 9584.614221] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9584.638524] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9589.747959] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9589.754952] Lustre: Skipped 3 previous similar messages [ 9594.335174] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9594.579849] Lustre: server umount lustre-OST0000 complete [ 9599.970636] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9599.996578] Lustre: Skipped 1 previous similar message [ 9605.071407] Lustre: server umount lustre-MDT0000 complete [ 9605.093216] LustreError: 293122:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9605.118260] LustreError: 293122:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9609.353465] LustreError: 293897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781234416 with bad export cookie 11184384174204657772 [ 9609.367092] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9609.370175] LustreError: 293897:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9609.842628] Lustre: server umount lustre-MDT0001 complete [ 9622.955309] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [ 9625.973732] Key type lgssc unregistered [ 9626.348817] LNet: 295098:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9626.367087] LNetError: 295098:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9627.429978] LNet: Removed LNI 192.168.201.136@tcp [ 9628.318262] Key type .llcrypt unregistered [ 9628.319757] Key type ._llcrypt unregistered [ 9655.131388] Key type ._llcrypt registered [ 9655.138204] Key type .llcrypt registered [ 9655.350995] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [ 9670.995280] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 9672.680140] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9672.710558] alg: No test for adler32 (adler32-zlib) [ 9673.773366] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [ 9673.951492] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [ 9675.727197] Key type lgssc registered [ 9677.253171] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9704.740530] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [ 9721.310488] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9721.348849] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9722.653227] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9722.689642] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9722.797956] Lustre: lustre-MDT0000: new disk, initializing [ 9722.914232] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9722.961239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9727.215920] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [ 9738.196567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9738.497774] Lustre: lustre-OST0000: new disk, initializing [ 9738.502517] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9738.507703] Lustre: Skipped 1 previous similar message [ 9738.522696] Lustre: 299084:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9738.609373] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9738.955777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9738.972273] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9739.001906] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9744.586990] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10517.472784] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10517.501882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10520.466763] Lustre: server umount lustre-MDT0000 complete [10524.583631] LustreError: 298290:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235331 with bad export cookie 3901125304629692852 [10524.612511] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10524.825170] Lustre: server umount lustre-OST0000 complete [10551.607227] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [10563.630137] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [10606.165804] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10606.442287] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10606.473812] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10606.578967] Lustre: lustre-MDT0000: new disk, initializing [10606.717509] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10606.735038] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10611.720282] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10623.188460] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10623.329780] Lustre: 303359:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10623.377835] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10623.382308] Lustre: Skipped 1 previous similar message [10623.460952] Lustre: lustre-MDT0001: new disk, initializing [10623.550484] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10623.586775] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10623.613875] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10628.568137] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10637.121865] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10641.149487] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10648.365967] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10648.691751] Lustre: lustre-OST0000: new disk, initializing [10648.698810] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10648.731785] Lustre: 305100:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10648.819987] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10654.276573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10654.284649] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10654.405458] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10654.783846] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10662.407641] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10667.217130] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10667.439444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10672.377000] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10672.601638] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10675.172195] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10675.173746] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10675.188515] LustreError: Skipped 1 previous similar message [10675.189957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10675.225676] Lustre: Skipped 1 previous similar message [10680.304045] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10680.314317] Lustre: Skipped 2 previous similar messages [10681.426696] Lustre: server umount lustre-OST0000 complete [10690.025220] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10690.040251] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10690.052940] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10692.642673] Lustre: server umount lustre-MDT0000 complete [10696.444967] LustreError: 304147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235503 with bad export cookie 3901125304632633671 [10696.451356] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10696.455184] LustreError: 304147:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10696.873939] Lustre: server umount lustre-MDT0001 complete [10700.509858] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10702.155276] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 23:38:27 (1781235507) [10710.276588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10711.194550] LustreError: 307266:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10711.218975] LustreError: 307266:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10711.273357] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10716.001501] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10716.639868] LustreError: 307267:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10721.768556] LustreError: 307266:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10725.070871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10725.618420] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10729.809856] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10735.957627] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10739.927350] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10747.243839] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10747.678562] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10753.710191] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10759.644629] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10767.433469] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10767.707536] Lustre: lustre-OST0001: new disk, initializing [10767.711146] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10767.715678] Lustre: 309762:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10767.910075] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10776.261944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10776.295861] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10776.384057] LustreError: 309980:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10776.397161] LustreError: 309980:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10777.452174] LustreError: 309980:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10777.470409] LustreError: 309980:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10778.474956] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10778.701572] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10780.518241] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10780.524932] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [10784.615541] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10784.629235] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [10792.460834] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10792.774431] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10792.867875] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10792.873722] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [10798.015888] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a16185c4000.ost_server_uuid 50 [10799.908438] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a16185c4000.ost_server_uuid in FULL state after 0 sec [10804.727947] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8a16185c4000.ost_server_uuid 50 [10807.121950] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8a16185c4000.ost_server_uuid in FULL state after 0 sec [10809.257768] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10809.288819] LustreError: 309993:0:(osp_precreate.c:970:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [10810.379281] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10825.194039] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10825.200887] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10825.220720] Lustre: Skipped 1 previous similar message [10825.237156] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10825.246093] Lustre: Skipped 1 previous similar message [10830.480448] Lustre: server umount lustre-OST0001 complete [10832.133528] LustreError: 308860:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.201.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10838.003079] LustreError: 308860:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10838.033738] LustreError: 308860:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10839.009836] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10839.020237] LustreError: Skipped 1 previous similar message [10839.022254] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10839.039631] Lustre: Skipped 1 previous similar message [10839.054864] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10839.065030] Lustre: Skipped 4 previous similar messages [10844.508344] Lustre: server umount lustre-OST0000 complete [10853.345494] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10853.348192] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10853.359460] LustreError: Skipped 1 previous similar message [10853.380183] Lustre: Skipped 2 previous similar messages [10856.032942] Lustre: server umount lustre-MDT0000 complete [10858.466608] LustreError: 307977:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10858.509068] LustreError: 307977:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10861.158156] LustreError: 307246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235668 with bad export cookie 3901125304632634553 [10861.163410] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10861.177714] LustreError: 307246:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [10861.776547] Lustre: server umount lustre-MDT0001 complete [10875.774326] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [10878.954284] Key type lgssc unregistered [10879.288276] LNet: 312208:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10879.311146] LNetError: 312208:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10879.330282] LNet: Removed LNI 192.168.201.136@tcp [10880.467955] Key type .llcrypt unregistered [10880.472288] Key type ._llcrypt unregistered [10911.062946] Key type ._llcrypt registered [10911.065392] Key type .llcrypt registered [10911.176847] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [10912.319266] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10912.417966] alg: No test for adler32 (adler32-zlib) [10913.806208] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [10914.034541] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [10915.815191] Key type lgssc registered [10917.327423] Lustre: Echo OBD driver; http://www.lustre.org/ [10926.490083] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10926.517077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10928.187241] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10933.736699] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10938.339440] LustreError: 313612:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10938.365183] LustreError: 313612:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [10939.366230] LustreError: 313613:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10940.294834] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10944.482907] LustreError: 313612:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10949.104341] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10949.678251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10956.483822] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10967.862130] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10968.346572] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10974.518669] LustreError: 315162:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10975.566722] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10976.181209] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [10980.836834] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10980.871252] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10983.148108] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10991.086833] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10991.103746] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10992.119305] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10992.551128] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10997.779523] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11001.259480] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11008.988427] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [11016.486618] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11016.837427] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11125.736130] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11125.767754] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11125.768420] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11125.783772] Lustre: Skipped 1 previous similar message [11128.083620] Lustre: server umount lustre-OST0001 complete [11130.854829] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11130.881329] LustreError: 315164:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11134.436687] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11134.443052] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11134.453171] Lustre: Skipped 1 previous similar message [11134.462266] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11135.974049] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11135.977444] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11141.091697] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11141.100465] Lustre: Skipped 2 previous similar messages [11146.227771] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11146.235411] Lustre: Skipped 1 previous similar message [11148.255133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11148.427510] Lustre: server umount lustre-OST0000 complete [11154.401276] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11154.410558] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11154.433383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11158.466888] Lustre: server umount lustre-MDT0000 complete [11162.837681] LustreError: 316922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781235969 with bad export cookie 3877612153904492539 [11162.838586] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11162.846895] LustreError: 316922:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11163.253354] Lustre: server umount lustre-MDT0001 complete [11174.589258] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [11177.743419] Key type lgssc unregistered [11177.999870] LNet: 318166:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11178.004615] LNetError: 318166:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11178.015857] LNet: Removed LNI 192.168.201.136@tcp [11178.721809] Key type .llcrypt unregistered [11178.724824] Key type ._llcrypt unregistered [11230.639488] Key type ._llcrypt registered [11230.641475] Key type .llcrypt registered [11230.746187] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [11232.818942] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11232.862431] alg: No test for adler32 (adler32-zlib) [11234.149688] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [11234.543051] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [11236.343654] Key type lgssc registered [11238.097239] Lustre: Echo OBD driver; http://www.lustre.org/ [11249.471254] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11249.491889] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11251.168141] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11256.032124] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11260.401643] LustreError: 320443:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11260.432986] LustreError: 320443:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11265.505659] LustreError: 320444:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11266.279427] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11266.869717] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11271.608430] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11274.391290] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11281.158764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11281.707722] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11287.796455] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11291.960057] LustreError: 321899:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11296.105405] LustreError: 321898:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11296.114607] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11296.130151] LustreError: 321898:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11297.082055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11297.424072] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11302.925016] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11304.587592] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11316.223521] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11323.496363] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11338.727112] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11338.728698] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11338.733904] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11338.759490] Lustre: Skipped 3 previous similar messages [11343.015094] Lustre: server umount lustre-MDT0000 complete [11343.847092] LustreError: 320444:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11343.882636] LustreError: 320444:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11347.395067] LustreError: 321547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236154 with bad export cookie 11674628211992084372 [11347.398398] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11347.412520] LustreError: 321547:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11348.054848] Lustre: server umount lustre-MDT0001 complete [11363.985197] Lustre: server umount lustre-OST0000 complete [11365.091506] Lustre: 319707:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781236155/real 1781236155] req@ffff9cae82090780 x1867761362992896/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781236171 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11365.134148] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11368.416067] Lustre: 319706:0:(client.c:2480:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1781236159/real 1781236159] req@ffff9caf856fe580 x1867761362993152/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1781236175 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11369.553146] Lustre: server umount lustre-OST0001 complete [11389.759804] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [11403.414722] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11404.131784] LustreError: 325625:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11404.266606] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11409.641863] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11419.057437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11419.766276] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11424.480678] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11427.775511] Lustre: 326732:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11436.041286] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11436.499824] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11443.744596] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11444.725275] LustreError: 327087:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11444.756226] LustreError: 327087:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11449.863339] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11453.895615] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11459.600214] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11461.773200] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11470.749384] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11474.976395] Lustre: 328567:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11480.040509] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11480.046789] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11480.050062] Lustre: Skipped 1 previous similar message [11480.065892] Lustre: Skipped 6 previous similar messages [11485.155336] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11485.173300] Lustre: Skipped 3 previous similar messages [11485.795265] Lustre: server umount lustre-MDT0000 complete [11489.848141] LustreError: 325607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236296 with bad export cookie 11674628211992086073 [11489.860237] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11489.870061] LustreError: 325607:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11490.276071] LustreError: 325622:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11490.276993] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11490.320298] LustreError: 325622:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11490.339373] Lustre: Skipped 2 previous similar messages [11490.630565] Lustre: server umount lustre-MDT0001 complete [11497.155192] Lustre: server umount lustre-OST0000 complete [11501.938275] Lustre: server umount lustre-OST0001 complete [11505.751099] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11507.492872] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11509.394277] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 23:51:54 (1781236314) [11534.601597] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [11544.983699] loop0: detected capacity change from 0 to 1099511627776 [11545.605713] SGI XFS with ACLs, security attributes, quota, no debug enabled [11545.616740] *** VALIDATE xfs *** [11545.640458] XFS (loop0): Mounting V5 Filesystem [11545.670213] XFS (loop0): Ending clean mount [11547.048247] loop1: detected capacity change from 0 to 18691697672192 [11676.848757] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [11687.002722] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [11723.312936] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11723.610538] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11723.645738] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11723.759607] Lustre: lustre-MDT0000: new disk, initializing [11723.875356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11723.879880] Lustre: Skipped 1 previous similar message [11723.902439] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11729.121218] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11743.224239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11743.461698] Lustre: 333657:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11743.612693] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11743.614709] Lustre: Skipped 1 previous similar message [11743.776396] Lustre: lustre-MDT0001: new disk, initializing [11744.104509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11744.142530] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11750.236453] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11758.867823] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11763.689987] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11770.963391] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11771.396757] Lustre: lustre-OST0000: new disk, initializing [11771.402085] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11771.416680] Lustre: 335404:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11772.691219] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11772.710770] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11772.850822] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11778.703634] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11786.808134] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11792.226625] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11792.533469] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11796.843631] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11797.131445] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11803.623303] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11803.627036] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11803.641491] LustreError: Skipped 1 previous similar message [11803.649616] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11803.662347] Lustre: Skipped 2 previous similar messages [11803.669457] Lustre: Skipped 1 previous similar message [11808.748431] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11808.759727] Lustre: Skipped 1 previous similar message [11813.343206] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11813.609259] Lustre: server umount lustre-OST0000 complete [11818.976546] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11818.994039] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11819.001310] Lustre: Skipped 1 previous similar message [11819.021593] Lustre: Skipped 1 previous similar message [11824.104309] LustreError: 333664:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11824.130631] LustreError: 333664:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11824.305373] Lustre: server umount lustre-MDT0000 complete [11828.750938] LustreError: 333650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236635 with bad export cookie 11674628211992087627 [11828.757674] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11828.762887] LustreError: 333650:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11829.177220] Lustre: server umount lustre-MDT0001 complete [11837.865476] XFS (loop0): Unmounting Filesystem [11842.870950] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 23:57:28 (1781236648) [11849.684443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11850.441949] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11850.453812] Lustre: Skipped 2 previous similar messages [11855.557669] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11864.339240] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11869.227934] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11875.895470] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11881.013483] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11889.819499] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11897.827671] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [11904.556767] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11916.270862] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11916.271468] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11916.297456] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11916.304833] LustreError: Skipped 1 previous similar message [11916.313881] Lustre: Skipped 1 previous similar message [11919.253662] Lustre: server umount lustre-OST0000 complete [11925.984875] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11929.943598] Lustre: server umount lustre-MDT0000 complete [11934.247592] LustreError: 337798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781236741 with bad export cookie 11674628211992088509 [11934.253295] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11934.264950] LustreError: 337798:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11934.627363] Lustre: server umount lustre-MDT0001 complete [11947.386295] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [11950.536284] Key type lgssc unregistered [11950.830940] LNet: 341089:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11950.840520] LNetError: 341089:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11951.910329] LNet: Removed LNI 192.168.201.136@tcp [11952.517587] Key type .llcrypt unregistered [11952.518946] Key type ._llcrypt unregistered [11982.699410] Key type ._llcrypt registered [11982.700742] Key type .llcrypt registered [11982.825604] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [11983.574390] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11983.590790] alg: No test for adler32 (adler32-zlib) [11984.599889] Lustre: Lustre: Build Version: 2.17.53_70_g68e5078 [11984.896753] LNet: Added LNI 192.168.201.136@tcp [8/256/0/180] [11986.663168] Key type lgssc registered [11987.868297] Lustre: Echo OBD driver; http://www.lustre.org/ [11997.171833] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11997.183113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11998.933758] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12003.062604] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12008.421982] LustreError: 342645:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12008.443479] LustreError: 342645:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12011.377208] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12011.831910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12016.600962] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12022.720409] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12027.225250] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12034.374503] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12034.771983] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12041.141089] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12047.282480] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12054.324832] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.36@tcp (stopping) [12054.498757] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12059.376760] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.36@tcp (stopping) [12059.392421] Lustre: Skipped 2 previous similar messages [12060.674251] Lustre: server umount lustre-MDT0001 complete [12064.505613] LustreError: 342642:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12069.628392] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12069.642448] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12072.771046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12072.828150] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12072.830171] Lustre: Found index 1 for lustre-MDT0001, updating log [12072.853863] Lustre: 342634:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12073.083166] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12078.093910] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12078.579447] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12079.623056] LustreError: 345817:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12079.637612] LustreError: 345817:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12079.645551] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12079.721821] LustreError: 345824:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.136@tcp: cfg command failed: rc = -17 [12079.738117] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12079.738117] [12385.775753] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12385.786161] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12385.814827] Lustre: Skipped 1 previous similar message [12385.821531] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12385.857230] Lustre: Skipped 3 previous similar messages [12390.705038] Lustre: server umount lustre-MDT0001 complete [12390.889603] LustreError: 342646:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12390.908846] LustreError: 342646:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12396.007866] LustreError: 343356:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12396.028460] LustreError: 343356:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12404.607915] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12404.815752] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12404.830747] Lustre: Found index 1 for lustre-MDT0001, updating log [12404.920598] Lustre: 342634:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12405.676866] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12410.879594] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12410.968481] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12412.418538] LustreError: 354047:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12412.443990] LustreError: 354047:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12412.463779] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12412.484138] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12412.642493] LustreError: 354064:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12412.657383] LustreError: 354064:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12412.683876] LustreError: 354064:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.136@tcp: cfg command failed: rc = -17 [12412.726085] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.136@tcp [12412.726085] [12718.048141] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12718.058797] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12718.060849] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12718.060868] Lustre: Skipped 1 previous similar message [12718.081396] Lustre: Skipped 2 previous similar messages [12723.173047] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12723.193632] Lustre: Skipped 3 previous similar messages [12723.694851] Lustre: server umount lustre-MDT0001 complete [12724.984272] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.36@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12725.005023] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [12734.042040] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12734.163106] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12734.177440] Lustre: Found index 1 for lustre-MDT0001, updating log [12734.208090] Lustre: 342634:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12734.575316] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12739.565756] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12739.584664] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [12742.686738] LustreError: 362504:0:(obd_mount.c:225:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12742.703333] LustreError: 362504:0:(tgt_mount.c:668:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12742.719721] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12742.725385] LustreError: 342637:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12742.773813] LustreError: 362515:0:(lod_lov.c:212:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12742.788800] LustreError: 362515:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.136@tcp: cfg command failed: rc = -114 [12742.796859] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12742.796859] [13067.237646] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13067.240046] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13067.255316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13067.267640] Lustre: Skipped 1 previous similar message [13069.508465] Lustre: server umount lustre-MDT0000 complete [13072.355087] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13072.377622] LustreError: 344196:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 7 previous similar messages [13073.514732] LustreError: 342627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781237880 with bad export cookie 5841582341260640575 [13073.519233] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13073.524564] LustreError: 342627:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13077.472939] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13077.488252] Lustre: Skipped 1 previous similar message [13080.008752] Lustre: server umount lustre-MDT0001 complete [13084.653964] Lustre: server umount lustre-OST0000 complete [13095.979380] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [13107.612390] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [13146.514493] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 00:19:12 (1781237952) [13153.674726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13153.949166] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13153.999784] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13154.091712] Lustre: lustre-MDT0000: new disk, initializing [13154.183524] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13154.206640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13158.477349] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [13169.099510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13169.260935] Lustre: 372868:0:(mgs_llog.c:1437:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13169.316195] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13169.321345] Lustre: Skipped 1 previous similar message [13169.381894] Lustre: lustre-MDT0001: new disk, initializing [13169.461834] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13169.492821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13169.503188] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13174.212320] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [13182.369610] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13186.609116] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13194.824481] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13195.308683] Lustre: lustre-OST0000: new disk, initializing [13195.312537] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13195.328884] Lustre: 374608:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13195.435110] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13202.551704] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_default_debug -1 all [13204.082105] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13204.096686] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13204.279830] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13211.395860] Lustre: DEBUG MARKER: oleg136-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13224.929358] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13224.934575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13224.950809] Lustre: Skipped 2 previous similar messages [13224.961709] Lustre: Skipped 6 previous similar messages [13229.322652] Lustre: server umount lustre-MDT0000 complete [13230.058757] LustreError: 372873:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13230.080874] LustreError: 372873:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [13233.860845] LustreError: 372858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1781238040 with bad export cookie 5841582341260642528 [13233.868768] LustreError: MGC192.168.201.136@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13233.883255] LustreError: 372858:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13234.417876] Lustre: server umount lustre-MDT0001 complete [13250.234930] Lustre: server umount lustre-OST0000 complete [13281.685326] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing set_hostid [13293.969219] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing load_modules_local [13330.082178] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13031 sec ========= 00:22:15 (1781238135) [13332.075605] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 00:22:17 (1781238137) === [13336.379590] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 00:22:21 (1781238141) === [13380.888607] Lustre: DEBUG MARKER: oleg136-server.virtnet: executing unload_modules_local [13385.099058] Key type lgssc unregistered [13385.461522] LNet: 380814:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13385.470745] LNetError: 380814:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13385.493288] LNet: Removed LNI 192.168.201.136@tcp [13386.399060] Key type .llcrypt unregistered [13386.401362] Key type ._llcrypt unregistered