[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 718823932 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.002074] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004016] kvm-guest: setup PV IPIs [ 0.007479] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008020] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010139] LSM: Security Framework initializing [ 0.011074] Yama: becoming mindful. [ 0.012046] SELinux: Initializing. [ 0.014029] *** VALIDATE selinux *** [ 0.024168] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.030272] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.031161] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032346] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033129] *** VALIDATE tmpfs *** [ 0.035268] *** VALIDATE proc *** [ 0.036228] *** VALIDATE cgroup *** [ 0.037013] *** VALIDATE cgroup2 *** [ 0.038273] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.039165] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.040010] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.041032] Spectre V2 : User space: Vulnerable [ 0.042010] Speculative Store Bypass: Vulnerable [ 0.045009] debug: unmapping init [mem 0xffffffff9ce59000-0xffffffff9ce60fff] [ 0.047276] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.048796] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.049025] ... version: 2 [ 0.050016] ... bit width: 48 [ 0.051016] ... generic registers: 4 [ 0.052014] ... value mask: 0000ffffffffffff [ 0.053020] ... max period: 00007fffffffffff [ 0.054014] ... fixed-purpose events: 3 [ 0.055012] ... event mask: 000000070000000f [ 0.056284] rcu: Hierarchical SRCU implementation. [ 0.058853] smp: Bringing up secondary CPUs ... [ 0.059809] x86: Booting SMP configuration: [ 0.060031] .... node #0, CPUs: #1 #2 #3 [ 0.072214] smp: Brought up 1 node, 4 CPUs [ 0.074019] smpboot: Max logical packages: 1 [ 0.075018] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.146023] node 0 deferred pages initialised in 69ms [ 0.150158] devtmpfs: initialized [ 0.151961] x86/mm: Memory block size: 128MB [ 0.156739] gcov: version magic: 0x41383552 [ 0.159035] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.160099] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.161336] pinctrl core: initialized pinctrl subsystem [ 0.162335] [ 0.163011] ************************************************************* [ 0.164018] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.165017] ** ** [ 0.166015] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.167015] ** ** [ 0.168016] ** This means that this kernel is built to expose internal ** [ 0.169015] ** IOMMU data structures, which may compromise security on ** [ 0.170017] ** your system. ** [ 0.171018] ** ** [ 0.172016] ** If you see this message and you are not debugging the ** [ 0.173016] ** kernel, report this immediately to your vendor! ** [ 0.174013] ** ** [ 0.175013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.176015] ************************************************************* [ 0.178026] NET: Registered protocol family 16 [ 0.179541] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.180200] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.181230] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.182689] cpuidle: using governor menu [ 0.186550] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.187729] PCI: Using configuration type 1 for base access [ 0.188125] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.198195] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.202021] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.206461] cryptd: max_cpu_qlen set to 1000 [ 0.209227] ACPI: Added _OSI(Module Device) [ 0.210015] ACPI: Added _OSI(Processor Device) [ 0.211018] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.212016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.216839] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.220528] ACPI: Interpreter enabled [ 0.221072] ACPI: PM: (supports S0 S3 S4 S5) [ 0.222015] ACPI: Using IOAPIC for interrupt routing [ 0.223105] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.224461] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.235976] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.236046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.237023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.238083] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.240692] acpiphp: Slot [2] registered [ 0.241331] acpiphp: Slot [5] registered [ 0.242264] acpiphp: Slot [6] registered [ 0.243134] acpiphp: Slot [7] registered [ 0.244146] acpiphp: Slot [8] registered [ 0.245156] acpiphp: Slot [9] registered [ 0.246172] acpiphp: Slot [10] registered [ 0.247170] acpiphp: Slot [3] registered [ 0.248114] acpiphp: Slot [4] registered [ 0.249133] acpiphp: Slot [11] registered [ 0.250123] acpiphp: Slot [12] registered [ 0.251126] acpiphp: Slot [13] registered [ 0.252118] acpiphp: Slot [14] registered [ 0.253146] acpiphp: Slot [15] registered [ 0.254116] acpiphp: Slot [16] registered [ 0.255098] acpiphp: Slot [17] registered [ 0.256572] acpiphp: Slot [18] registered [ 0.257204] acpiphp: Slot [19] registered [ 0.258125] acpiphp: Slot [20] registered [ 0.259315] acpiphp: Slot [21] registered [ 0.260161] acpiphp: Slot [22] registered [ 0.261322] acpiphp: Slot [23] registered [ 0.262337] acpiphp: Slot [24] registered [ 0.263232] acpiphp: Slot [25] registered [ 0.264307] acpiphp: Slot [26] registered [ 0.265133] acpiphp: Slot [27] registered [ 0.266135] acpiphp: Slot [28] registered [ 0.267189] acpiphp: Slot [29] registered [ 0.268199] acpiphp: Slot [30] registered [ 0.269098] acpiphp: Slot [31] registered [ 0.270068] PCI host bridge to bus 0000:00 [ 0.271087] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.272232] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.273028] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.274024] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.275033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.276029] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.277434] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.279298] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.281322] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.287965] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.291068] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.292022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.293019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.294018] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.295952] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.300077] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.303047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.305789] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.315015] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.336013] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.343017] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.354702] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.366016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.376000] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.403015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.417583] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.425017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.434020] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.454017] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.466785] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.475019] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.486016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.511019] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.522398] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.532015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.538016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.559021] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.591518] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.600016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.608020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.623019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.642329] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.654019] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.672015] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.700031] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.715000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.720081] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.727858] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.731684] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.734432] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.738277] iommu: Default domain type: Passthrough [ 0.740585] SCSI subsystem initialized [ 0.743288] ACPI: bus type USB registered [ 0.745138] usbcore: registered new interface driver usbfs [ 0.749131] usbcore: registered new interface driver hub [ 0.752118] usbcore: registered new device driver usb [ 0.754489] pps_core: LinuxPPS API ver. 1 registered [ 0.757016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.761237] PTP clock support registered [ 0.765148] EDAC MC: Ver: 3.0.0 [ 0.767362] PCI: Using ACPI for IRQ routing [ 0.770491] NetLabel: Initializing [ 0.772019] NetLabel: domain hash size = 128 [ 0.775019] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.778175] NetLabel: unlabeled traffic allowed by default [ 0.781756] vgaarb: loaded [ 0.785023] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.788015] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.797203] clocksource: Switched to clocksource kvm-clock [ 0.960667] VFS: Disk quotas dquot_6.6.0 [ 0.964572] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.966996] *** VALIDATE ramfs *** [ 0.968450] *** VALIDATE hugetlbfs *** [ 0.970356] pnp: PnP ACPI init [ 0.973297] pnp: PnP ACPI: found 6 devices [ 0.990155] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.994587] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.997781] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.000644] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.003647] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.006666] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.010258] NET: Registered protocol family 2 [ 1.013332] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.020248] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.025461] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.031804] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.036457] TCP: Hash tables configured (established 65536 bind 65536) [ 1.040250] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.044109] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.047782] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.051594] NET: Registered protocol family 1 [ 1.055699] RPC: Registered named UNIX socket transport module. [ 1.058389] RPC: Registered udp transport module. [ 1.061881] RPC: Registered tcp transport module. [ 1.064320] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.067790] NET: Registered protocol family 44 [ 1.070218] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.072922] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.075245] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.078213] PCI: CLS 0 bytes, default 64 [ 1.080508] Unpacking initramfs... [ 3.570919] debug: unmapping init [mem 0xffff9974bcc54000-0xffff9974bffbffff] [ 3.575943] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.578765] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 3.582445] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 4.216704] Initialise system trusted keyrings [ 4.222197] Key type blacklist registered [ 4.291223] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.311171] zbud: loaded [ 4.316088] *** VALIDATE nfs *** [ 4.318175] *** VALIDATE nfs4 *** [ 4.320223] pstore: using deflate compression [ 4.325685] Platform Keyring initialized [ 4.512586] NET: Registered protocol family 38 [ 4.515591] Key type asymmetric registered [ 4.517670] Asymmetric key parser 'x509' registered [ 4.520252] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.525324] io scheduler mq-deadline registered [ 4.527613] io scheduler kyber registered [ 4.529748] io scheduler bfq registered [ 4.532224] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.536057] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.539700] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.543457] ACPI: Power Button [PWRF] [ 4.558394] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.570437] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.585493] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.595088] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.615633] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.647017] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 4.680397] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 4.687638] Non-volatile memory driver v1.3 [ 4.689575] Linux agpgart interface v0.103 [ 4.725659] virtio_blk virtio1: [vda] 145912 512-byte logical blocks (74.7 MB/71.2 MiB) [ 4.729545] vda: detected capacity change from 0 to 74706944 [ 4.754498] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 4.758897] vdb: detected capacity change from 0 to 1073741824 [ 4.773967] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.777342] vdc: detected capacity change from 0 to 2621440000 [ 4.794392] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 4.798173] vdd: detected capacity change from 0 to 2621440000 [ 4.813692] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.817410] vde: detected capacity change from 0 to 4294967296 [ 4.839130] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 4.844286] vdf: detected capacity change from 0 to 4294967296 [ 4.856201] libphy: Fixed MDIO Bus: probed [ 4.866726] usbcore: registered new interface driver usbserial_generic [ 4.870704] usbserial: USB Serial support registered for generic [ 4.874140] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 4.881269] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 4.884146] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 4.887575] mousedev: PS/2 mouse device common for all mice [ 4.893263] rtc_cmos 00:05: RTC can wake from S4 [ 4.896667] rtc_cmos 00:05: registered as rtc0 [ 4.900638] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 4.904711] intel_pstate: CPU model not supported [ 4.908070] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 4.916200] hid: raw HID events driver (C) Jiri Kosina [ 4.918641] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 4.923637] usbcore: registered new interface driver usbhid [ 4.933851] usbhid: USB HID core driver [ 4.936599] drop_monitor: Initializing network drop monitor service [ 4.940319] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 4.941658] Initializing XFRM netlink socket [ 4.942280] NET: Registered protocol family 10 [ 4.948773] Segment Routing with IPv6 [ 4.957132] NET: Registered protocol family 17 [ 4.960023] mpls_gso: MPLS GSO support [ 4.969140] RAS: Correctable Errors collector initialized. [ 4.972413] AVX version of gcm_enc/dec engaged. [ 4.975402] AES CTR mode by8 optimization enabled [ 5.120414] sched_clock: Marking stable (5120204261, 0)->(6838490762, -1718286501) [ 5.126337] registered taskstats version 1 [ 5.128258] Loading compiled-in X.509 certificates [ 5.130555] zswap: loaded using pool lzo/zbud [ 5.179639] Key type big_key registered [ 5.209420] Key type encrypted registered [ 5.215264] ima: No TPM chip found, activating TPM-bypass! [ 5.222364] ima: Allocated hash algorithm: sha1 [ 5.225590] ima: No architecture policies found [ 5.228261] evm: Initialising EVM extended attributes: [ 5.231667] evm: security.selinux [ 5.234039] evm: security.ima [ 5.236586] evm: security.capability [ 5.243066] evm: HMAC attrs: 0x1 [ 5.255111] rtc_cmos 00:05: setting system clock to 2026-08-11 06:59:14 UTC (1786431554) [ 5.274276] debug: unmapping init [mem 0xffffffff9de03000-0xffffffff9dffffff] [ 5.285260] debug: unmapping init [mem 0xffffffff9cb82000-0xffffffff9ce58fff] [ 5.297073] Write protecting the kernel read-only data: 28672k [ 5.304620] debug: unmapping init [mem 0xffffffff9b203000-0xffffffff9b3fffff] [ 5.311946] debug: unmapping init [mem 0xffffffff9bb14000-0xffffffff9bbfffff] [ 5.401156] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.422892] systemd[1]: Detected virtualization kvm. [ 5.426828] systemd[1]: Detected architecture x86-64. [ 5.430433] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.474741] systemd[1]: No hostname configured. [ 5.477378] systemd[1]: Set hostname to . [ 5.480526] random: systemd: uninitialized urandom read (16 bytes read) [ 5.484319] systemd[1]: Initializing machine ID from random generator. [ 5.559667] random: ln: uninitialized urandom read (6 bytes read) [ 5.747071] random: systemd: uninitialized urandom read (16 bytes read) [ 5.753124] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ 5.763921] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 5.771785] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Swap. [ OK ] Reached target Local File Systems. [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Sockets. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.357266] device-mapper: uevent: version 1.0.3 [ 7.360860] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 9.405182] random: fast init done [ 9.499867] virtio_net virtio0 ens2: renamed from eth0 [ 9.694228] scsi host0: ata_piix [ 9.941975] scsi host1: ata_piix [ 9.944073] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 9.947831] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 14.845509] random: crng init done [ 14.852809] random: 7 urandom warning(s) missed due to ratelimiting [ 17.228521] dracut-initqueue[572]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.450296] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local File Systems. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 21.946273] printk: systemd: 26 output lines suppressed due to ratelimiting [ 22.641106] SELinux: Disabled at runtime. [ 22.753272] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 22.763749] systemd[1]: Detected virtualization kvm. [ 22.766113] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 24.444587] systemd[1]: initrd-switch-root.service: Succeeded. [ 24.449305] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 24.458253] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 24.466520] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 24.475245] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 24.487782] systemd[1]: Starting Journal Service... Starting Journal Service... [ 24.508532] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... [ OK ] Reached target Slices. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Created slice system-sshd\x2dkeygen.slice. Activating swap /dev/disk/by-label/SWAP... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on udev Control Socket. Starting Apply Kernel Variables... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 24.964530] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Reached target rpc_pipefs.target. Starting udev Coldplug all Devices... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ OK ] Started Journal Service. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 26.714995] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 27.887340] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 28.120916] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 29.011521] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 29.126769] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit)[ 32.753437] Key type dns_resolver registered [ 33.182646] NFS: Registering the id_resolver key type [ 33.188617] Key type id_resolver registered [ 33.194271] Key type id_legacy registered [** ] A start job is running for Configur…-only root support (9s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started D-Bus System Message Bus. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Network Manager... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Started OpenSSH server daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Login Service. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg309-server login: [ 79.503051] hrtimer: interrupt took 9088464 ns [ 91.986814] libcfs: loading out-of-tree module taints kernel. [ 92.040737] Key type ._llcrypt registered [ 92.045423] Key type .llcrypt registered [ 92.137155] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 111.136090] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 114.126855] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 114.147705] alg: No test for adler32 (adler32-zlib) [ 116.235869] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 117.303824] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 119.166525] Key type lgssc registered [ 122.169600] Lustre: Echo OBD driver; http://www.lustre.org/ [ 139.336768] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 177.779447] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 191.794373] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 191.819448] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 193.001431] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 193.108130] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 193.169662] Lustre: lustre-MDT0000: new disk, initializing [ 193.261208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 193.302703] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 197.306783] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.271992] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 209.360841] Lustre: 6510:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 209.385801] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 209.390971] Lustre: Skipped 1 previous similar message [ 209.472966] Lustre: lustre-MDT0001: new disk, initializing [ 209.559229] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 209.619712] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 209.635152] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 213.074655] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 218.391997] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 227.961942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 228.160798] Lustre: lustre-OST0000: new disk, initializing [ 228.165688] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 228.175888] Lustre: 8448:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 228.280885] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 234.738403] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 238.238888] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 238.248387] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 238.406994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 250.482774] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 250.580897] Lustre: lustre-OST0001: new disk, initializing [ 250.583659] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 250.588513] Lustre: 9521:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 250.641855] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 256.017961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 256.026030] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 256.088338] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 256.721743] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 269.022796] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 281.433630] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 287.563844] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing check_logdir /tmp/testlogs/ [ 293.839428] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing yml_node [ 300.038708] Lustre: DEBUG MARKER: Client: 2.17.56.51 [ 302.798216] Lustre: DEBUG MARKER: MDS: 2.17.56.51 [ 305.422717] Lustre: DEBUG MARKER: OSS: 2.17.56.51 [ 307.132469] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Aug 11 03:04:14 EDT 2026 [ 326.293743] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 328.106881] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 348.128107] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 348.128594] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 348.128677] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 348.137082] Lustre: Skipped 1 previous similar message [ 348.163648] Lustre: Skipped 3 previous similar messages [ 350.363800] Lustre: server umount lustre-MDT0000 complete [ 353.761973] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786431903 with bad export cookie 1827271690700704404 [ 353.779667] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 353.784123] LustreError: 6503:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 358.368936] LustreError: 9161:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 358.387018] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 358.392919] LustreError: 9161:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 358.422259] Lustre: Skipped 2 previous similar messages [ 358.431472] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 358.440570] Lustre: Skipped 1 previous similar message [ 363.489500] LustreError: 9161:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 363.490420] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 363.506042] LustreError: 9161:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 363.511320] Lustre: Skipped 1 previous similar message [ 364.513250] LustreError: 6514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 364.535328] LustreError: 6514:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 368.095546] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 368.273942] Lustre: server umount lustre-MDT0001 complete [ 371.526958] Lustre: server umount lustre-OST0000 complete [ 375.761837] Lustre: server umount lustre-OST0001 complete [ 382.003370] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 390.164792] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 427.146715] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 427.442539] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 427.472278] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 427.552236] Lustre: lustre-MDT0000: new disk, initializing [ 427.612726] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 427.626874] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 431.429715] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 442.590761] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 442.719442] Lustre: 15080:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 442.771675] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 442.775148] Lustre: Skipped 1 previous similar message [ 442.826141] Lustre: lustre-MDT0001: new disk, initializing [ 442.929250] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 443.047411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 443.097321] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 446.761566] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 455.687304] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 461.210910] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 468.694300] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 469.108920] Lustre: lustre-OST0000: new disk, initializing [ 469.113794] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 469.119865] Lustre: 17046:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 469.322208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 470.624811] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 470.630569] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 471.108561] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 475.676337] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 484.874711] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 489.059663] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 489.414272] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 494.024913] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 494.208755] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 496.615566] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 496.621571] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 496.648677] Lustre: Skipped 1 previous similar message [ 496.654096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 496.661790] Lustre: Skipped 2 previous similar messages [ 501.755940] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 501.774372] Lustre: Skipped 1 previous similar message [ 502.165572] Lustre: server umount lustre-OST0000 complete [ 506.848970] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 506.860798] Lustre: Skipped 2 previous similar messages [ 511.663595] Lustre: server umount lustre-MDT0000 complete [ 511.968256] LustreError: 15092:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 511.977400] LustreError: 15092:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 514.943179] LustreError: 15073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432064 with bad export cookie 1827271690700705944 [ 514.946061] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 514.952268] LustreError: 15073:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 515.221860] Lustre: server umount lustre-MDT0001 complete [ 519.602970] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:07:47 (1786432067) [ 526.607342] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 526.972336] LustreError: 19311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 527.033962] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 531.098620] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 537.569583] LustreError: 19311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 537.585576] LustreError: 19311:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 538.536191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 542.197353] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 548.737261] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 554.062862] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 559.902148] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 566.270589] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 574.015405] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 578.382499] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 585.704424] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 585.749378] LustreError: Skipped 1 previous similar message [ 585.758121] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 585.774863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 585.781353] Lustre: Skipped 2 previous similar messages [ 590.665266] Lustre: server umount lustre-MDT0000 complete [ 593.388472] LustreError: 20559:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 593.441880] LustreError: 20559:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 594.684219] LustreError: 19293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432143 with bad export cookie 1827271690700706826 [ 594.693226] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 594.917885] Lustre: server umount lustre-MDT0001 complete [ 607.973870] Lustre: server umount lustre-OST0000 complete [ 629.296534] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 640.029947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 640.347542] LustreError: 23691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 640.460355] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 640.466658] Lustre: Skipped 2 previous similar messages [ 644.921473] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 652.408029] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 656.720572] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 659.285780] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 666.357106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 671.893715] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 679.314662] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.452175] Lustre: lustre-OST0001: new disk, initializing [ 679.455612] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 679.461282] Lustre: 26008:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 680.825331] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 680.835722] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 680.940474] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 685.493280] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 694.527667] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 697.447369] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 709.090192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 709.102758] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 709.110766] Lustre: Skipped 2 previous similar messages [ 709.118802] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 709.125343] Lustre: Skipped 3 previous similar messages [ 714.384697] Lustre: server umount lustre-MDT0000 complete [ 715.748446] LustreError: 23692:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 715.764900] LustreError: 23692:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 717.859918] LustreError: 23671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432267 with bad export cookie 1827271690700707666 [ 717.861895] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 717.869058] LustreError: 23671:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 718.286235] Lustre: server umount lustre-MDT0001 complete [ 730.949704] Lustre: server umount lustre-OST0000 complete [ 744.561896] Lustre: server umount lustre-OST0001 complete [ 758.603255] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 768.573726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 768.895932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 768.904744] Lustre: Skipped 3 previous similar messages [ 773.071427] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 780.353904] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 784.525939] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 787.171851] Lustre: 30174:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 792.854850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 798.677416] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 808.142251] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 813.748592] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 819.920216] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 823.473610] Lustre: 32049:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 828.918477] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 828.921689] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 828.940308] Lustre: Skipped 5 previous similar messages [ 828.969454] Lustre: Skipped 7 previous similar messages [ 834.467422] Lustre: server umount lustre-MDT0000 complete [ 838.393789] LustreError: 29005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432387 with bad export cookie 1827271690700708968 [ 838.399687] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 838.404262] LustreError: 29005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 839.184950] Lustre: server umount lustre-MDT0001 complete [ 853.696957] Lustre: server umount lustre-OST0000 complete [ 855.647236] Lustre: 3647:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786432388/real 1786432388] req@ffff997402d38000 x1873209370896000/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786432404 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 857.114383] Lustre: server umount lustre-OST0001 complete [ 867.590918] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:13:35 (1786432415) [ 894.043629] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 902.444314] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 902.716442] LustreError: 34851:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 902.744629] LustreError: 34851:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 907.028888] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 915.352383] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 919.271414] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 922.136646] Lustre: 35999:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 927.959313] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 934.890966] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 942.583745] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 948.701885] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 955.741949] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 958.954648] Lustre: 37876:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 963.562895] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 963.567389] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 963.592932] Lustre: Skipped 4 previous similar messages [ 963.597132] Lustre: Skipped 5 previous similar messages [ 969.113472] Lustre: server umount lustre-MDT0000 complete [ 972.166091] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432521 with bad export cookie 1827271690700710074 [ 972.168425] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 972.174361] LustreError: 34833:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 972.412662] Lustre: server umount lustre-MDT0001 complete [ 986.295906] Lustre: server umount lustre-OST0000 complete [ 999.423963] Lustre: server umount lustre-OST0001 complete [ 1007.656252] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:15:55 (1786432555) [ 1022.535355] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1032.434655] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1032.764158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1032.779589] Lustre: Skipped 7 previous similar messages [ 1036.898358] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1043.859443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1047.773957] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1050.798774] Lustre: 41168:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1056.219195] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1062.497197] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1070.585199] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1076.590102] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1083.563939] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1086.863383] Lustre: 43044:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1087.555772] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1087.562533] Lustre: Skipped 1 previous similar message [ 1091.568309] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1091.571096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1091.587598] Lustre: Skipped 4 previous similar messages [ 1091.598042] Lustre: Skipped 7 previous similar messages [ 1096.872883] Lustre: server umount lustre-MDT0000 complete [ 1100.853384] LustreError: 43046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432650 with bad export cookie 1827271690700711180 [ 1100.861303] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1100.866028] LustreError: 43046:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1107.396352] Lustre: server umount lustre-MDT0001 complete [ 1115.155590] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1126.373826] LustreError: 3643:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff99753e5e4e00 x1873209370994560/t0(0) o250->MGC192.168.203.109@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1130.438367] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1135.920992] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1137.819559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1142.192535] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1143.283965] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1143.289883] Lustre: Skipped 1 previous similar message [ 1148.863131] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1150.539726] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1156.296764] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1157.970113] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1163.769194] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1167.981129] Lustre: server umount lustre-MDT0000 complete [ 1168.877065] LustreError: 41532:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1168.906571] LustreError: 41532:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 36 previous similar messages [ 1171.362538] LustreError: 45582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432720 with bad export cookie 1827271690700712279 [ 1171.375155] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1171.378132] LustreError: 45582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1171.620257] Lustre: server umount lustre-MDT0001 complete [ 1185.290395] Lustre: server umount lustre-OST0000 complete [ 1198.445297] Lustre: server umount lustre-OST0001 complete [ 1206.854777] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:19:14 (1786432754) [ 1220.174861] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1229.359268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1233.312897] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1240.359310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1245.045906] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1247.229512] Lustre: 48807:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1253.966952] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1260.625810] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1268.037619] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1274.857508] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1282.672447] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1287.501789] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1299.426451] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1299.468670] Lustre: Skipped 8 previous similar messages [ 1299.839203] Lustre: server umount lustre-MDT0000 complete [ 1303.379796] LustreError: 50693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786432852 with bad export cookie 1827271690700713098 [ 1303.383040] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1303.390269] LustreError: 50693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1303.678224] Lustre: server umount lustre-MDT0001 complete [ 1316.893749] Lustre: server umount lustre-OST0000 complete [ 1319.907252] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786432853/real 1786432853] req@ffff9974033d6d80 x1873209371081600/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786432869 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1320.826232] Lustre: server umount lustre-OST0001 complete [ 1329.334375] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:21:17 (1786432877) [ 1332.041277] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1333.772442] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:21:21 (1786432881) [ 1343.269846] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 1350.668259] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1372.780743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1372.948670] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1372.994876] Lustre: lustre-MDT0000: new disk, initializing [ 1373.052756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1376.756788] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1384.458919] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1390.296679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1390.504720] Lustre: lustre-OST0000: new disk, initializing [ 1390.508505] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1390.513085] Lustre: Skipped 1 previous similar message [ 1390.516969] Lustre: 54882:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1392.133067] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1392.149856] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1392.191912] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1396.197844] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1405.462294] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1407.591612] Lustre: DEBUG MARKER: create test files [ 1433.063605] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1433.082515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1433.094642] Lustre: Skipped 14 previous similar messages [ 1438.886612] Lustre: server umount lustre-OST0000 complete [ 1443.045724] Lustre: server umount lustre-MDT0000 complete [ 1453.292715] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 1455.343643] Key type lgssc unregistered [ 1455.550268] LNet: 56516:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1455.562453] LNetError: 56516:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1455.571794] LNet: Removed LNI 192.168.203.109@tcp [ 1456.546235] Key type .llcrypt unregistered [ 1456.550225] Key type ._llcrypt unregistered [ 1483.389676] Key type ._llcrypt registered [ 1483.392843] Key type .llcrypt registered [ 1483.507394] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1484.214458] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1484.249059] alg: No test for adler32 (adler32-zlib) [ 1485.321742] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 1485.490735] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 1487.210049] Key type lgssc registered [ 1488.198097] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1494.838052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1494.856369] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1496.173873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1500.759621] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1507.847201] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1515.498810] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1515.986219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1521.177929] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1521.249775] Lustre: 59232:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 515 < left 544, rollback = 9 [ 1521.260189] Lustre: 59232:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/0, destroy: 0/0/0 [ 1521.264975] Lustre: 59232:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1521.269949] Lustre: 59232:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1521.278837] Lustre: 59232:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1521.297628] Lustre: 59232:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1523.177474] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1530.119328] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1536.564946] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1542.111192] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1551.840649] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1551.849493] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1553.921978] Lustre: server umount lustre-OST0000 complete [ 1557.721685] Lustre: server umount lustre-MDT0000 complete [ 1569.404155] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 1571.447741] Key type lgssc unregistered [ 1571.924663] LNet: 60842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1571.929853] LNetError: 60842:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1571.943236] LNet: Removed LNI 192.168.203.109@tcp [ 1572.751944] Key type .llcrypt unregistered [ 1572.753765] Key type ._llcrypt unregistered [ 1601.414177] Key type ._llcrypt registered [ 1601.419740] Key type .llcrypt registered [ 1601.498255] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1602.552222] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1602.622414] alg: No test for adler32 (adler32-zlib) [ 1603.603943] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 1603.780519] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 1605.431154] Key type lgssc registered [ 1606.857920] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1615.101697] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1615.116683] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1616.425843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1620.344867] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1627.058341] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1635.752029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1636.044248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1641.197907] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1641.290937] Lustre: 63672:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1641.300585] Lustre: 63672:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1641.313228] Lustre: 63672:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1641.322397] Lustre: 63672:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1641.340687] Lustre: 63672:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1641.347796] Lustre: 63672:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1642.774632] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1649.400721] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1655.574245] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1667.042298] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1667.050246] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1667.059414] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1669.007603] Lustre: server umount lustre-OST0000 complete [ 1672.609632] Lustre: server umount lustre-MDT0000 complete [ 1683.899549] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 1686.749401] Key type lgssc unregistered [ 1687.238957] LNet: 65231:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1687.251118] LNetError: 65231:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1687.292786] LNet: Removed LNI 192.168.203.109@tcp [ 1687.996685] Key type .llcrypt unregistered [ 1687.998940] Key type ._llcrypt unregistered [ 1706.477252] Key type ._llcrypt registered [ 1706.494950] Key type .llcrypt registered [ 1706.591763] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 1720.497439] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1721.858644] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1721.920495] alg: No test for adler32 (adler32-zlib) [ 1723.003679] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 1723.199839] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 1724.831171] Key type lgssc registered [ 1725.830051] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1758.489490] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1758.531893] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1759.973278] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1759.994378] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1760.056418] Lustre: lustre-MDT0000: new disk, initializing [ 1760.124309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1760.139347] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1764.248822] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1773.279350] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1773.489753] Lustre: 68385:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1773.566870] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1773.571241] Lustre: Skipped 1 previous similar message [ 1773.767386] Lustre: lustre-MDT0001: new disk, initializing [ 1773.854805] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1773.878787] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1773.888406] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1778.450319] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1787.695621] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1793.182870] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1800.258506] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1800.541538] Lustre: lustre-OST0000: new disk, initializing [ 1800.544575] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1800.550234] Lustre: 70355:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1800.702588] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1806.486984] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1806.969444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1806.976514] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1807.084604] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1816.932660] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1821.220357] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1821.575727] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1825.930474] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1826.085611] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1827.814416] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1827.816898] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1827.829183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1827.842066] LustreError: Skipped 1 previous similar message [ 1832.940489] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1832.944119] Lustre: Skipped 2 previous similar messages [ 1838.051297] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1838.056044] Lustre: Skipped 1 previous similar message [ 1842.143187] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1842.232795] Lustre: server umount lustre-OST0000 complete [ 1845.728373] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1845.737236] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1845.752200] Lustre: Skipped 1 previous similar message [ 1845.759204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1848.308864] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1853.411590] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1853.425165] Lustre: Skipped 3 previous similar messages [ 1860.063221] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1860.176781] Lustre: server umount lustre-MDT0000 complete [ 1864.524279] LustreError: 68378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786433413 with bad export cookie 319679657357879857 [ 1864.536636] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1864.538988] LustreError: 68378:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1864.791171] Lustre: server umount lustre-MDT0001 complete [ 1874.873268] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 03:30:22 (1786433422) [ 1884.762413] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1921.892728] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 1932.163687] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 1966.256024] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1966.422824] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1966.442167] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1966.513378] Lustre: lustre-MDT0000: new disk, initializing [ 1966.594514] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1966.619325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1970.362679] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1979.051664] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1979.114337] Lustre: 76376:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1979.166353] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1979.170188] Lustre: Skipped 1 previous similar message [ 1979.224318] Lustre: lustre-MDT0001: new disk, initializing [ 1979.287566] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1979.311552] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1979.321878] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1983.125735] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 1992.826606] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1997.663619] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2003.941361] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2004.207719] Lustre: lustre-OST0000: new disk, initializing [ 2004.215748] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2004.232985] Lustre: 78344:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2004.379959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2010.456097] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2012.728723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2012.735241] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2012.840685] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2019.071229] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2023.662918] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2024.162405] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2028.779866] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2029.130848] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2033.636693] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2033.640688] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2033.646965] LustreError: Skipped 1 previous similar message [ 2033.651588] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2033.663479] Lustre: Skipped 1 previous similar message [ 2033.671324] Lustre: Skipped 2 previous similar messages [ 2037.163162] Lustre: server umount lustre-OST0000 complete [ 2043.874078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2047.483203] Lustre: server umount lustre-MDT0000 complete [ 2048.993734] LustreError: 76383:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2049.010586] LustreError: 76383:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2050.822536] LustreError: 78343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786433600 with bad export cookie 319679657357880739 [ 2050.826629] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2050.841855] LustreError: 78343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2051.063577] Lustre: server umount lustre-MDT0001 complete [ 2060.026779] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 03:33:27 (1786433607) [ 2065.453996] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2065.780510] LustreError: 80753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2066.029560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2070.575223] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2071.025484] LustreError: 80754:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2076.130158] LustreError: 80753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2078.857605] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2083.609832] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2091.056806] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2096.697517] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2103.496514] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2103.708768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2103.712115] Lustre: Skipped 1 previous similar message [ 2108.981714] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2115.868323] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2118.758310] Lustre: 83424:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 217baa6a-a3c9-45cd-8c52-57bc8e54b563 at adminstrative request [ 2121.191850] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2121.204824] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2131.451619] Lustre: MGS: Client 217baa6a-a3c9-45cd-8c52-57bc8e54b563 (at 0@lo) reconnecting [ 2131.465670] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x46fbaf462bc3115 to 0x46fbaf462bc33b5 [ 2131.539231] Lustre: MGC192.168.203.109@tcp: Connection restored to 0@lo (at 0@lo) [ 2156.062032] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2156.156640] Lustre: lustre-OST0001: new disk, initializing [ 2156.160178] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2156.167486] Lustre: 83815:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2162.338345] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2166.323768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2166.336948] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2166.473376] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2171.851496] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2176.993134] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2176.999722] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2177.014576] LustreError: Skipped 1 previous similar message [ 2177.042723] Lustre: Skipped 1 previous similar message [ 2177.049915] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2177.154185] Lustre: Skipped 4 previous similar messages [ 2181.217856] Lustre: server umount lustre-OST0001 complete [ 2182.115211] LustreError: 82574:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2182.149143] LustreError: 82574:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2186.722430] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2192.356487] LustreError: 83445:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2192.372746] LustreError: 83445:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2199.007129] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2199.273065] Lustre: server umount lustre-OST0000 complete [ 2202.592554] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2202.599733] Lustre: Skipped 3 previous similar messages [ 2207.200917] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2210.808977] Lustre: server umount lustre-MDT0000 complete [ 2212.834470] LustreError: 80750:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2212.853347] LustreError: 80750:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2214.282404] LustreError: 80733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786433763 with bad export cookie 319679657357882293 [ 2214.285508] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2214.292625] LustreError: 80733:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2214.738376] Lustre: server umount lustre-MDT0001 complete [ 2222.311900] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 03:36:10 (1786433770) [ 2223.754223] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2225.529603] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 03:36:13 (1786433773) [ 2227.318130] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2228.782281] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 03:36:16 (1786433776) [ 2230.911631] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2232.796388] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 03:36:20 (1786433780) [ 2234.493926] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2236.679401] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 03:36:24 (1786433784) [ 2238.853756] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2244.371551] loop: module loaded [ 2244.392139] loop0: detected capacity change from 0 to 4294967296 [ 2244.419021] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2245.725044] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2249.460708] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2253.350811] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2277.163957] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2277.230608] LustreError: 87942:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2277.239340] LustreError: 87942:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2277.256755] LustreError: 87942:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2277.271274] LustreError: 87942:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2293.512967] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 2302.593142] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 2336.908059] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2337.099933] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2337.117307] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2337.171547] Lustre: lustre-MDT0000: new disk, initializing [ 2337.222253] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2337.229123] Lustre: Skipped 1 previous similar message [ 2337.242440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2341.575702] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2350.708206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2350.817638] Lustre: 90810:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2351.015355] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2356.246379] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2365.445866] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2370.292813] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2377.043482] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2377.250642] Lustre: 92777:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2382.889208] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2383.019623] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2383.270625] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2392.045659] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2396.380708] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2396.945079] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2402.305802] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2402.754558] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2408.419760] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2408.441332] Lustre: Skipped 1 previous similar message [ 2408.455645] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2408.461390] Lustre: Skipped 10 previous similar messages [ 2410.819325] Lustre: server umount lustre-OST0000 complete [ 2417.642667] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2420.679470] Lustre: server umount lustre-MDT0000 complete [ 2423.776253] LustreError: 90821:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2423.793475] LustreError: 90821:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2424.218435] LustreError: 90801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786433973 with bad export cookie 319679657357882909 [ 2424.225183] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2424.230248] LustreError: 90801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2424.567663] Lustre: server umount lustre-MDT0001 complete [ 2432.863536] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 03:39:40 (1786433980) [ 2438.853309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2444.102637] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2452.212856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2456.322326] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2463.299357] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2467.793232] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2473.977733] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2474.211001] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2474.216968] Lustre: Skipped 4 previous similar messages [ 2479.837532] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2487.814569] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2491.611830] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2491.866887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2496.435426] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2496.723955] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2503.261054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2503.415695] Lustre: lustre-OST0001: new disk, initializing [ 2503.418565] Lustre: Skipped 2 previous similar messages [ 2503.421594] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2503.425103] Lustre: Skipped 3 previous similar messages [ 2503.429518] Lustre: 98556:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2509.089539] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2509.461084] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2509.477528] Lustre: Skipped 2 previous similar messages [ 2509.494764] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2509.615102] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2517.972915] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2522.078728] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2522.346469] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2526.144530] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2526.618420] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2546.240585] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2568.225032] Lustre: Failing over lustre-MDT0000 [ 2568.451792] Lustre: lustre-MDT0000: Not available for connect from 192.168.203.9@tcp (stopping) [ 2568.465604] Lustre: Skipped 5 previous similar messages [ 2568.813664] Lustre: server umount lustre-MDT0000 complete [ 2570.210658] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2570.268125] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2570.290244] Lustre: Skipped 3 previous similar messages [ 2570.297950] LustreError: 95189:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2570.308281] LustreError: 95189:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2587.682863] Lustre: 66611:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786434120/real 1786434120] req@00000000731b6dc1 x1873211054887296/t0(0) o400->MGC192.168.203.109@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786434136 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2587.713901] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2592.713267] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2592.715701] LDISKFS-fs (dm-0): recovery complete [ 2592.720581] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2596.832417] LustreError: 66608:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@000000002ca69515 x1873211054895616/t0(0) o250->MGC192.168.203.109@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2597.100081] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2599.149739] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2599.152790] Lustre: 99958:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2601.469639] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2602.467877] Lustre: 95189:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2602.473734] Lustre: 95189:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2602.486256] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2602.560071] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2602.895731] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2603.220741] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2603.242090] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2603.579513] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2603.584350] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2603.589696] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2603.597439] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2604.240028] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2604.248802] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2604.879670] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2604.891352] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 3 previous similar messages [ 2605.897633] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2605.921312] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2606.295939] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2606.304531] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2606.951126] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2606.961915] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2610.112877] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2610.124080] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2610.456735] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2610.468780] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2611.119126] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2611.124978] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2618.245433] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2618.256836] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2618.584130] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2618.592564] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2619.236155] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2619.242269] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2634.249632] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2634.279487] Lustre: 101406:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 49 previous similar messages [ 2634.616407] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2634.623777] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2635.311147] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2635.325138] LustreError: 101406:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 49 previous similar messages [ 2659.039197] Lustre: 101406:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2659.049691] Lustre: 101406:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2659.062174] Lustre: 101406:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 216556e1-9750-42b2-8d86-92650ef967db@192.168.203.9@tcp [ 2659.074946] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2659.077996] Lustre: 101406:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@000000008098fd11 x1873211040009856/t0(8589934771) o101->216556e1-9750-42b2-8d86-92650ef967db@192.168.203.9@tcp:29/0 lens 592/0 e 0 to 0 dl 1786434219 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2659.093620] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2659.104547] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2659.113418] Lustre: lustre-MDT0000: Denying connection for new client 216556e1-9750-42b2-8d86-92650ef967db (at 192.168.203.9@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2659.134856] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2659.136774] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2659.142919] Lustre: Skipped 2 previous similar messages [ 2659.173309] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2659.179339] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2672.096331] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2678.105549] Lustre: server umount lustre-OST0000 complete [ 2695.135301] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2695.212173] Lustre: server umount lustre-OST0001 complete [ 2699.783264] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2699.796316] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2699.805713] Lustre: Skipped 8 previous similar messages [ 2699.812949] Lustre: Skipped 9 previous similar messages [ 2705.294849] Lustre: server umount lustre-MDT0000 complete [ 2709.557493] LustreError: 95169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786434258 with bad export cookie 319679657357934121 [ 2709.563054] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2709.571378] LustreError: 95169:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2710.204611] Lustre: server umount lustre-MDT0001 complete [ 2719.066688] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:44:26 (1786434266) [ 2726.537520] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2726.543403] Lustre: *** cfs_fail_loc=197, val=0*** [ 2742.047657] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2747.167479] LustreError: 103381:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.109@tcp: failed processing log, type 4: rc = -110 [ 2752.347853] Lustre: *** cfs_fail_loc=197, val=0*** [ 2752.350213] Lustre: Skipped 65 previous similar messages [ 2772.895312] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2772.901163] Lustre: Skipped 2 previous similar messages [ 2778.992633] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2786.358797] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2788.268801] Lustre: server umount lustre-OST0000 complete [ 2797.262961] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:45:45 (1786434345) [ 2818.405266] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2818.868198] LustreError: 105001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2818.876329] LustreError: 105001:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 46 previous similar messages [ 2823.623268] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2832.110762] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2835.672926] LustreError: 105032:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2835.683558] LustreError: 105032:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2835.689391] LustreError: 105032:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2835.963415] Lustre: server umount lustre-MDT0000 complete [ 2867.351053] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 2875.804546] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 2908.090305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2908.282365] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2908.307403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2908.381497] Lustre: lustre-MDT0000: new disk, initializing [ 2908.462918] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2913.177740] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2923.113783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2923.231053] Lustre: 109207:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2923.607085] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2928.370822] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2936.786271] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2941.966909] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2949.193072] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2949.364518] Lustre: 111175:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2955.081779] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 2959.604661] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2964.324134] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2968.687736] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2968.874631] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2973.472322] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2973.848862] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2979.809502] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2979.810209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2979.811210] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2979.811215] LustreError: Skipped 1 previous similar message [ 2979.818289] Lustre: Skipped 1 previous similar message [ 2979.846789] Lustre: Skipped 3 previous similar messages [ 2981.819276] Lustre: server umount lustre-OST0000 complete [ 2991.799643] Lustre: server umount lustre-MDT0000 complete [ 2994.837976] LustreError: 109198:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786434544 with bad export cookie 319679657357944719 [ 2994.847020] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2995.185373] Lustre: server umount lustre-MDT0001 complete [ 3004.431237] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:49:12 (1786434552) [ 3012.382316] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 3015.057782] Key type lgssc unregistered [ 3015.397839] LNet: 113664:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3015.403240] LNetError: 113664:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3015.417756] LNet: Removed LNI 192.168.203.109@tcp [ 3016.110548] Key type .llcrypt unregistered [ 3016.113396] Key type ._llcrypt unregistered [ 3032.482256] Key type ._llcrypt registered [ 3032.487021] Key type .llcrypt registered [ 3032.873428] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 3048.064163] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3050.217246] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3050.259341] alg: No test for adler32 (adler32-zlib) [ 3051.501510] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 3051.769087] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 3053.459113] Key type lgssc registered [ 3054.520715] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3093.119957] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3093.183962] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3094.382710] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3094.430122] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3094.482713] Lustre: lustre-MDT0000: new disk, initializing [ 3094.542895] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3094.614033] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3098.671887] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3110.270483] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3110.488305] Lustre: lustre-OST0000: new disk, initializing [ 3110.494891] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3110.498688] Lustre: Skipped 1 previous similar message [ 3110.502921] Lustre: 117842:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3110.554650] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3112.925469] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3112.941099] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3112.969917] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3117.107491] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3127.660883] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3127.834980] Lustre: lustre-OST0001: new disk, initializing [ 3127.838360] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3127.843449] Lustre: 118707:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3128.037981] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3135.527379] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3137.298875] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3137.381272] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3137.450427] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3147.746991] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3147.752529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3147.761980] Lustre: Skipped 1 previous similar message [ 3147.770950] Lustre: Skipped 1 previous similar message [ 3151.122552] Lustre: server umount lustre-MDT0000 complete [ 3157.527148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3162.434615] LustreError: 117031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786434711 with bad export cookie 15318412612390899806 [ 3162.438611] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3162.461542] LustreError: 117031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3162.618545] Lustre: server umount lustre-OST0000 complete [ 3166.072959] Lustre: server umount lustre-OST0001 complete [ 3175.289211] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 03:52:02 (1786434722) [ 3196.962692] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 03:52:24 (1786434744) [ 3207.143990] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3231.269352] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:52:58 (1786434778) [ 3267.258657] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 3275.819385] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3310.453574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3310.638993] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3310.658692] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3310.710655] Lustre: lustre-MDT0000: new disk, initializing [ 3310.759204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3310.781966] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3314.343299] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3323.911803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3324.021512] Lustre: 126456:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3324.069501] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3324.074544] Lustre: Skipped 1 previous similar message [ 3324.139712] Lustre: lustre-MDT0001: new disk, initializing [ 3324.227974] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3324.250506] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3324.263772] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3328.808869] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3338.487834] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3343.460584] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3349.852193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3350.070103] Lustre: lustre-OST0000: new disk, initializing [ 3350.075681] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3350.083198] Lustre: 128426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3350.272228] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3355.871139] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3356.361452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3356.410394] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3356.670169] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3365.024326] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3370.289514] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3370.451616] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3374.975204] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3375.207629] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3377.121832] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3377.125086] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3377.130688] LustreError: Skipped 1 previous similar message [ 3377.155305] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3377.159232] Lustre: Skipped 1 previous similar message [ 3382.241533] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3382.245748] Lustre: Skipped 1 previous similar message [ 3383.030876] Lustre: server umount lustre-OST0000 complete [ 3387.363485] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3387.380045] Lustre: Skipped 1 previous similar message [ 3387.387497] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3387.400970] Lustre: Skipped 1 previous similar message [ 3392.482736] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3392.490455] Lustre: Skipped 1 previous similar message [ 3392.937582] Lustre: server umount lustre-MDT0000 complete [ 3395.895404] LustreError: 126450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786434945 with bad export cookie 15318412612390901031 [ 3395.908362] LustreError: 126450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3395.912271] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3396.244430] Lustre: server umount lustre-MDT0001 complete [ 3404.702192] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:55:52 (1786434952) [ 3411.209326] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3411.649752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3415.538482] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3417.063094] LustreError: 130838:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3417.099732] LustreError: 130838:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3421.176199] LustreError: 130839:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3424.137575] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3428.113641] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3434.605898] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3440.754146] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3447.628986] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3447.873925] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3447.879710] Lustre: Skipped 1 previous similar message [ 3453.607740] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3459.746705] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3463.760898] Lustre: *** cfs_fail_loc=159, val=0*** [ 3479.358603] Lustre: lustre-MDT0000: Client ca135ca6-0b99-40cf-b9a6-b986b963ccd6 (at 192.168.203.9@tcp) reconnecting [ 3480.700286] Lustre: *** cfs_fail_loc=159, val=0*** [ 3480.707714] Lustre: Skipped 5 previous similar messages [ 3496.255141] Lustre: lustre-MDT0000: Client ca135ca6-0b99-40cf-b9a6-b986b963ccd6 (at 192.168.203.9@tcp) reconnecting [ 3500.000386] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3500.003604] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3500.010484] Lustre: Skipped 1 previous similar message [ 3500.014350] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3502.049469] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3506.120353] Lustre: server umount lustre-OST0000 complete [ 3511.264396] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3511.268282] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3511.274527] Lustre: Skipped 1 previous similar message [ 3515.278735] Lustre: server umount lustre-MDT0000 complete [ 3517.409058] LustreError: 130835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3517.421099] LustreError: 130835:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3518.701438] LustreError: 133556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435067 with bad export cookie 15318412612390901913 [ 3518.713293] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3518.723022] LustreError: 133556:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3519.216029] Lustre: server umount lustre-MDT0001 complete [ 3528.799120] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 3531.021635] Key type lgssc unregistered [ 3531.294240] LNet: 134614:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3531.309972] LNetError: 134614:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3531.331838] LNet: Removed LNI 192.168.203.109@tcp [ 3532.122169] Key type .llcrypt unregistered [ 3532.126652] Key type ._llcrypt unregistered [ 3554.142913] Key type ._llcrypt registered [ 3554.145200] Key type .llcrypt registered [ 3554.241291] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3555.200280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3555.216392] alg: No test for adler32 (adler32-zlib) [ 3556.378423] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 3556.557393] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 3558.312749] Key type lgssc registered [ 3559.381319] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3566.199584] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3566.225286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3567.580587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3571.579836] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3576.807154] LustreError: 136031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3576.823333] LustreError: 136031:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3578.918388] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3579.218080] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3583.189854] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3590.804435] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3597.193952] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3604.830526] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3605.045488] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3610.094738] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3612.235508] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3617.179811] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3629.684891] Lustre: *** cfs_fail_loc=159, val=0*** [ 3645.181373] Lustre: lustre-MDT0000: Client 0ee14cb0-f493-439f-8502-cce9f24312b8 (at 192.168.203.9@tcp) reconnecting [ 3647.222167] Lustre: *** cfs_fail_loc=159, val=0*** [ 3663.675914] Lustre: lustre-MDT0001: Client 0ee14cb0-f493-439f-8502-cce9f24312b8 (at 192.168.203.9@tcp) reconnecting [ 3664.991888] Lustre: *** cfs_fail_loc=159, val=0*** [ 3664.996962] Lustre: Skipped 3 previous similar messages [ 3681.581500] Lustre: lustre-MDT0001: Client 0ee14cb0-f493-439f-8502-cce9f24312b8 (at 192.168.203.9@tcp) reconnecting [ 3684.817190] Lustre: 139077:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3687.414691] Lustre: *** cfs_fail_loc=159, val=0*** [ 3687.426854] Lustre: Skipped 6 previous similar messages [ 3703.109913] Lustre: lustre-MDT0000: Client 27f311be-3241-46b8-95af-04bb35f9051f (at 192.168.203.9@tcp) reconnecting [ 3704.808365] Lustre: *** cfs_fail_loc=159, val=0*** [ 3704.814602] Lustre: Skipped 8 previous similar messages [ 3719.952960] Lustre: lustre-MDT0000: Client 27f311be-3241-46b8-95af-04bb35f9051f (at 192.168.203.9@tcp) reconnecting [ 3721.401292] Lustre: 139387:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3742.183235] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3742.183811] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3742.207229] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3742.229213] Lustre: Skipped 1 previous similar message [ 3747.179099] Lustre: server umount lustre-OST0000 complete [ 3752.417025] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3752.419119] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3752.424913] Lustre: Skipped 2 previous similar messages [ 3756.571579] Lustre: server umount lustre-MDT0000 complete [ 3757.537449] LustreError: 136032:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3757.576283] LustreError: 136032:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3760.482241] LustreError: 136013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435309 with bad export cookie 16614191188743217150 [ 3760.487499] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3760.503442] LustreError: 136013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3761.018248] Lustre: server umount lustre-MDT0001 complete [ 3770.667623] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 3773.126259] Key type lgssc unregistered [ 3773.371953] LNet: 140474:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3773.376957] LNetError: 140474:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3773.387849] LNet: Removed LNI 192.168.203.109@tcp [ 3774.117532] Key type .llcrypt unregistered [ 3774.119707] Key type ._llcrypt unregistered [ 3794.303352] Key type ._llcrypt registered [ 3794.305934] Key type .llcrypt registered [ 3794.393588] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3795.408163] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3795.498891] alg: No test for adler32 (adler32-zlib) [ 3796.622234] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 3796.803522] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 3798.439132] Key type lgssc registered [ 3799.340627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3805.521099] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3805.534726] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3807.060258] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3811.218225] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3817.447339] LustreError: 141868:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3817.468164] LustreError: 141868:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3818.201764] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3818.485711] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3822.182791] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3829.270137] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3834.461243] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3840.519287] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3840.790820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3842.880134] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3847.071817] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3852.069869] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3853.436757] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3857.802482] Lustre: 144449:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3860.078981] Lustre: 144499:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3861.259599] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3861.967714] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3864.600431] Lustre: 144644:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3865.252175] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3865.935875] Lustre: 144742:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3868.640235] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3868.644230] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3868.654502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3872.739036] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3872.742405] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3872.743379] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3872.753043] Lustre: Skipped 1 previous similar message [ 3874.662991] Lustre: server umount lustre-OST0000 complete [ 3877.856891] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3877.860590] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3877.873884] Lustre: Skipped 1 previous similar message [ 3882.984426] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3882.996048] Lustre: Skipped 2 previous similar messages [ 3883.928354] Lustre: server umount lustre-MDT0000 complete [ 3886.886664] LustreError: 141849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435436 with bad export cookie 3074699249371586445 [ 3886.918805] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3887.520521] Lustre: server umount lustre-MDT0001 complete [ 3896.334907] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 3898.503396] Key type lgssc unregistered [ 3898.737063] LNet: 145803:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3898.742373] LNetError: 145803:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3898.755406] LNet: Removed LNI 192.168.203.109@tcp [ 3899.419143] Key type .llcrypt unregistered [ 3899.421614] Key type ._llcrypt unregistered [ 3920.911503] Key type ._llcrypt registered [ 3920.915934] Key type .llcrypt registered [ 3921.031241] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 3922.460357] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3922.493965] alg: No test for adler32 (adler32-zlib) [ 3923.607530] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 3923.797452] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 3925.527933] Key type lgssc registered [ 3926.357615] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3933.924673] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3933.939680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3935.409690] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3939.283095] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3944.932402] LustreError: 147196:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3944.968585] LustreError: 147196:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3949.716044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3950.081619] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3955.594957] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3962.435193] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3967.273690] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3973.276699] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3973.584265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3975.687857] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 3978.642986] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 3980.800678] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 3985.264259] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3991.327878] Lustre: *** cfs_fail_loc=159, val=0*** [ 4007.681582] Lustre: lustre-MDT0000: Client b9a923f7-b1e8-4aeb-8775-c64c13efd467 (at 192.168.203.9@tcp) reconnecting [ 4013.029962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4013.033044] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4013.036163] LustreError: Skipped 2 previous similar messages [ 4013.045495] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4016.627184] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4016.636090] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4016.647807] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4017.961983] Lustre: server umount lustre-OST0000 complete [ 4021.732903] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4021.736174] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4021.752163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4021.752171] Lustre: Skipped 1 previous similar message [ 4021.752868] Lustre: Skipped 1 previous similar message [ 4026.850293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4026.855350] Lustre: Skipped 2 previous similar messages [ 4029.810515] Lustre: server umount lustre-MDT0000 complete [ 4031.969038] LustreError: 147192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4031.978674] LustreError: 147192:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4033.188669] LustreError: 149050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435582 with bad export cookie 16401165897419712857 [ 4033.194977] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4033.199473] LustreError: 149050:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4033.444754] Lustre: server umount lustre-MDT0001 complete [ 4042.763301] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 4045.100756] Key type lgssc unregistered [ 4045.311792] LNet: 150864:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4045.317266] LNetError: 150864:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4046.373880] LNet: Removed LNI 192.168.203.109@tcp [ 4047.174978] Key type .llcrypt unregistered [ 4047.178425] Key type ._llcrypt unregistered [ 4067.935741] Key type ._llcrypt registered [ 4067.941406] Key type .llcrypt registered [ 4068.177442] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 4069.324872] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4069.425281] alg: No test for adler32 (adler32-zlib) [ 4070.389831] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 4070.791759] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 4072.487328] Key type lgssc registered [ 4073.251652] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4080.079157] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4080.093355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4081.469417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4085.404285] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4090.856546] LustreError: 152260:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4090.889386] LustreError: 152260:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4092.816782] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4093.093356] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4097.340841] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4104.560474] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4108.677444] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4114.738533] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4115.167462] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4118.251195] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4121.168940] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4126.461649] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4128.080854] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4147.814674] Lustre: lustre-MDT0001: evicting 8e4a0734-2736-48ac-bb02-bf1290929ab4 (at 192.168.203.9@tcp) by administrative request [ 4177.889743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4177.891932] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4177.904510] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4181.682715] Lustre: server umount lustre-OST0000 complete [ 4188.143690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4188.150223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4188.152851] Lustre: Skipped 2 previous similar messages [ 4188.161270] Lustre: Skipped 2 previous similar messages [ 4191.592647] Lustre: server umount lustre-MDT0000 complete [ 4193.257461] LustreError: 152261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4193.271117] LustreError: 152261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4195.661483] LustreError: 152239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435744 with bad export cookie 4082080578642052098 [ 4195.671883] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4195.675874] LustreError: 152239:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4196.062184] Lustre: server umount lustre-MDT0001 complete [ 4205.881796] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 4208.109293] Key type lgssc unregistered [ 4208.329844] LNet: 156331:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4208.341658] LNetError: 156331:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4208.363422] LNet: Removed LNI 192.168.203.109@tcp [ 4209.016668] Key type .llcrypt unregistered [ 4209.018611] Key type ._llcrypt unregistered [ 4279.533809] Key type ._llcrypt registered [ 4279.537718] Key type .llcrypt registered [ 4279.645177] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 4280.113773] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4280.161496] alg: No test for adler32 (adler32-zlib) [ 4281.208227] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 4281.412444] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 4283.080413] Key type lgssc registered [ 4284.060802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4290.806960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4290.819752] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4292.142907] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4295.606936] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4302.307367] LustreError: 158819:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4302.422982] LustreError: 158819:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4303.567418] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4303.930559] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4308.051987] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4315.149858] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4320.161899] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4326.397317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4326.623714] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4329.710077] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4331.860849] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4336.885523] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4337.596775] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4345.313372] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4345.319127] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4345.328198] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4347.364145] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4347.371644] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4347.377137] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4347.404281] Lustre: Skipped 1 previous similar message [ 4351.347855] Lustre: server umount lustre-OST0000 complete [ 4355.041292] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4355.047000] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4355.054243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4357.601272] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4357.602711] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4360.890580] Lustre: server umount lustre-MDT0000 complete [ 4362.720360] LustreError: 160065:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4362.735915] LustreError: 160065:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4363.994306] LustreError: 158801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786435913 with bad export cookie 1681217821662397925 [ 4364.006178] LustreError: 158801:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4364.011353] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4364.425444] Lustre: server umount lustre-MDT0001 complete [ 4373.988462] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 4376.169172] Key type lgssc unregistered [ 4376.498798] LNet: 162364:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4376.503936] LNetError: 162364:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4376.515215] LNet: Removed LNI 192.168.203.109@tcp [ 4377.234457] Key type .llcrypt unregistered [ 4377.236420] Key type ._llcrypt unregistered [ 4398.929339] Key type ._llcrypt registered [ 4398.931315] Key type .llcrypt registered [ 4399.132244] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 4400.241585] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4400.412102] alg: No test for adler32 (adler32-zlib) [ 4401.479174] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 4401.668284] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 4403.303163] Key type lgssc registered [ 4404.404271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4417.097925] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:12:45 (1786435965) [ 4430.583844] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 4438.732066] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 4468.193850] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4468.212349] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4469.364930] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4469.385644] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4469.451771] Lustre: lustre-MDT0000: new disk, initializing [ 4469.542388] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4469.561850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4474.469754] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4484.470268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4484.900546] Lustre: 166623:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4485.082606] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4485.088634] Lustre: Skipped 1 previous similar message [ 4485.136320] Lustre: lustre-MDT0001: new disk, initializing [ 4485.187383] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4485.221022] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4485.227643] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4489.499114] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4498.858309] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4504.657809] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4512.547759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4512.812276] Lustre: lustre-OST0000: new disk, initializing [ 4512.815671] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4512.820862] Lustre: 168596:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4512.925093] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4521.234350] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4521.288036] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4521.752968] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4521.806051] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4531.375913] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4541.927036] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4541.940689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4541.946195] Lustre: Skipped 1 previous similar message [ 4541.966151] Lustre: Skipped 1 previous similar message [ 4547.059785] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4547.069621] Lustre: Skipped 1 previous similar message [ 4547.671421] Lustre: server umount lustre-OST0000 complete [ 4551.651201] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4551.664172] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4551.695112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4557.303629] Lustre: server umount lustre-MDT0000 complete [ 4560.654069] LustreError: 166617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786436109 with bad export cookie 3763202079527054962 [ 4560.655151] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4560.741567] LustreError: 166617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4560.999334] Lustre: server umount lustre-MDT0001 complete [ 4571.944361] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 4574.418214] Key type lgssc unregistered [ 4574.655900] LNet: 170501:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4574.660851] LNetError: 170501:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4574.679304] LNet: Removed LNI 192.168.203.109@tcp [ 4575.877082] Key type .llcrypt unregistered [ 4575.879235] Key type ._llcrypt unregistered [ 4601.941375] Key type ._llcrypt registered [ 4601.944261] Key type .llcrypt registered [ 4602.107251] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 4604.072732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4604.124707] alg: No test for adler32 (adler32-zlib) [ 4605.166901] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 4605.384976] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 4607.127273] Key type lgssc registered [ 4607.940625] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4616.005886] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4616.026156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4617.490785] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4621.397742] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4626.948033] LustreError: 171918:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4626.996052] LustreError: 171918:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4630.375824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4631.047596] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4636.055732] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4643.264877] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4648.308161] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4654.201770] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4654.645677] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4660.678393] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 4661.896270] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4667.623407] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4674.871890] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4674.884534] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4674.923330] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4674.934304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4674.948589] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4676.374908] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4676.386378] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4676.394853] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4676.399912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4676.413294] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4677.145299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.082343] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4678.089495] Lustre: Skipped 1 previous similar message [ 4678.107986] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4678.115838] Lustre: Skipped 1 previous similar message [ 4678.120086] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4678.124646] Lustre: Skipped 1 previous similar message [ 4678.127670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4678.134776] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4678.140460] Lustre: Skipped 1 previous similar message [ 4679.253717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4680.012164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.188969] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4681.194540] Lustre: Skipped 2 previous similar messages [ 4681.213575] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4681.222665] Lustre: Skipped 2 previous similar messages [ 4681.227086] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4681.234754] Lustre: Skipped 2 previous similar messages [ 4681.238901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4681.249623] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4681.256228] Lustre: Skipped 2 previous similar messages [ 4682.088056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4682.878892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4683.580178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4684.428518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.135054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.933447] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4685.937772] Lustre: Skipped 5 previous similar messages [ 4685.950599] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4685.957937] Lustre: Skipped 5 previous similar messages [ 4685.961237] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4685.967747] Lustre: Skipped 5 previous similar messages [ 4685.974939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4685.983366] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4685.994646] Lustre: Skipped 5 previous similar messages [ 4686.725885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4688.088724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4689.196746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4690.271574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4691.165403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4692.167794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.210588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4693.954279] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4693.958255] Lustre: Skipped 7 previous similar messages [ 4693.974748] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4693.985856] Lustre: Skipped 7 previous similar messages [ 4693.991092] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4693.996788] Lustre: Skipped 7 previous similar messages [ 4694.001621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4694.014155] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4694.021257] Lustre: Skipped 7 previous similar messages [ 4694.737116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4695.769316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4696.610336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4697.803709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4698.707942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4699.580542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4700.310918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.051724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4701.904124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4702.719568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4703.532136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.277701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4704.824500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4705.698722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4706.431924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4707.707170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4708.692071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4709.788629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.715976] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4710.722718] Lustre: Skipped 18 previous similar messages [ 4710.744679] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4710.761473] Lustre: Skipped 18 previous similar messages [ 4710.769461] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4710.775864] Lustre: Skipped 18 previous similar messages [ 4710.780648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4710.790430] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4710.800840] Lustre: Skipped 18 previous similar messages [ 4711.618816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4712.620434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4713.503781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.082336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4714.766372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4715.616944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4717.666909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4719.418543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4720.434514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4721.145441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.107738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4722.750260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4723.742932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4724.618790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4725.761759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4727.375740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4728.638703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.050448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4730.995291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4732.305510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.376234] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4733.406856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4733.409784] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4734.193051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4735.389478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4736.451669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4737.345798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4738.327919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4739.358617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.176813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4740.928697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4741.687258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4742.348035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.417213] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4743.429483] Lustre: Skipped 31 previous similar messages [ 4743.475580] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4743.546264] Lustre: Skipped 31 previous similar messages [ 4743.549423] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4743.553541] Lustre: Skipped 31 previous similar messages [ 4743.556703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4743.564122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4743.568038] Lustre: Skipped 31 previous similar messages [ 4744.396336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.263297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4745.920789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4746.896868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4747.627069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4748.483581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4749.166448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.125756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4750.796572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4751.547533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4752.307405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4753.182296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.041352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4754.859519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4755.791398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4756.455579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4757.439105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4758.276955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.172139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4759.983265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4760.632677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4761.293645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4762.535542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4763.677871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4764.737487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4765.443841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4766.280431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4767.293618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4768.687325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4769.611987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4770.644882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4771.335953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4772.324450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4773.339494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.192714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4774.868603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4775.481408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.202884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4776.946646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4777.684499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4778.532191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4779.422556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4780.746885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4781.548444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4782.339481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4783.087533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4784.295242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.134423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4785.945946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4786.761145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4787.563121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4788.406790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.166902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4789.791304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4790.528510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4791.311037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4792.207217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4793.019482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.066910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4794.844076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.859451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.630334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4797.362224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.510345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4799.334708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.015735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.667329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.424975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.585384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.639366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.720386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4805.774135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.799525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.566990] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4807.571670] Lustre: Skipped 73 previous similar messages [ 4807.584782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4807.592542] Lustre: Skipped 73 previous similar messages [ 4807.596180] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4807.599851] Lustre: Skipped 73 previous similar messages [ 4807.603329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4807.612716] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4807.626865] Lustre: Skipped 73 previous similar messages [ 4808.431785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.341896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.258483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4811.069888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.073388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.811359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.219577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4815.060269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.038951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.858614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.747240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.497248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.788821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.630474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.800949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.987694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.964948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.729642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.402059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.792894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.197149] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4828.216959] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4828.234903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.140493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.104552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.960294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4831.873178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.834925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.672434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.524871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.489835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.719617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.617575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.258120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.226269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.817401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.486501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.338642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.189813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.631859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.810704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.828252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.663457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.731745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.610271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.383418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.222573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.028140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.144839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.884897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.014775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.843655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.072495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.971847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.019645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.747070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.775595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.067645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.098523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.247141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.907721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4864.876603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.819508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.466590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.384388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.156249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.283725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.123651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.332232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.536070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.587254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.502431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.315535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.204859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.211443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.543555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.399170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.458457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.617933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.730770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.451891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.327703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.702990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.841783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.691772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.640440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.805205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.771290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.046960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.079622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.028928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.827202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.536514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.105972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.970960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.888490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.834576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.565950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.201460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.329622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.314397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.097507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.884572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.021177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.862839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.037236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.079957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.157966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.937544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.623517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.356724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.215235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.185771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.028240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.185567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.309958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.097123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.955143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.581520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.832069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.634260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.736383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.649897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.895760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.744211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.470441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.292586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4928.408212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.652429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.445361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.141993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.133195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.059822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.433285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.229195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.370458] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4936.379293] Lustre: Skipped 133 previous similar messages [ 4936.399939] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4936.411362] Lustre: Skipped 133 previous similar messages [ 4936.416816] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4936.422918] Lustre: Skipped 133 previous similar messages [ 4936.430419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.442652] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4936.449747] Lustre: Skipped 133 previous similar messages [ 4937.653642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.499429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.643278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.854701] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4940.869316] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4940.885415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.700646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.953321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.120391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.839224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.674977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.815042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.135220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.957983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.932163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.585760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.796650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.966558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.225751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.052906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.827953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.709970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.500414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.523405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.331896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.379620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.124869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.883516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.685996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.778595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.503497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.326608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.283919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.120258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.368669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.650987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.417965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.119472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.952423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.187975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.093618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.439142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.229709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.210987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.531581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.470654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.641890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.622832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.692946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.453129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.354650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.764210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.727838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.501861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.489530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.421785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.157732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.015356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.985032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.653318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.457275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.188805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.884690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.549204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.334668] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 4998.345453] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 4998.355445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.670657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.486347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.331473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.662996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.408029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.427996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.175972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.920439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.723241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.470191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.277744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.409512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.771728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.827336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.629948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.305202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.713482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.692141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.863185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.573470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.486304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.614207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.316202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.311308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.035462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.795377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.568261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.433093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.189938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.072806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.929901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.936675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.666651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.539792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.502670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.483453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.351183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.211201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.006508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.247741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.905126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.772275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.438783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.137264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.044609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.018876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.934619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.938264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.589168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.303860] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5044.347312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.352471] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5045.336265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.439429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.439663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.336493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.401667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.214989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.093367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.018166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.150169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.146137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.140233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.088475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.874414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.854212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.903538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.879197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.713420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.668518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.168016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.865538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.662990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.502895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.344109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.155580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.405882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.346313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.328274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.391181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.262354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.247409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.293357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.506517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.338413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.226266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.003514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.696603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.865355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.747385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.386249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.122160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.247805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.101525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.045623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.256312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.456618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.207125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.028890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.008310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.926070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.558717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.305569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.114272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.847967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.607227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.441961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.533901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.290943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.481177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.426360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.450988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.395728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.130079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.079275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.945735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.914517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.766212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.032656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.918716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.961182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.735851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.072709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.083773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.123647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.916564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.084452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.682850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.733976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.366698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.205048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.427107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.739319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.691993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.592896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.341359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.319118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.530297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.654818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.644689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.402193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.357426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.350847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.424944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.316838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.552854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.777448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.544506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.523097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.491839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.650645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.378155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.285147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.059407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.141538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.900453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.633081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.515773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.334231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.353665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.486529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.360988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.331506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.198603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.364334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.096803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.019914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.864665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.653401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.721687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.567687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.464375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.655809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.497277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.473524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.155851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.192227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.125483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.811943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.629805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.643984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.299205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.053356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.054977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.997654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.966710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.745344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.445793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.376973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.571906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.295553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.310708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.285424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.419377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.488056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.259349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.261032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.059898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.864714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.115683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.031825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.140105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.377169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.064296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.143503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.240346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.217297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.810807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.554440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.408986] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5192.413511] Lustre: Skipped 270 previous similar messages [ 5192.424920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5192.432583] Lustre: Skipped 270 previous similar messages [ 5192.436286] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5192.440635] Lustre: Skipped 270 previous similar messages [ 5192.444572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.455093] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5192.459688] Lustre: Skipped 270 previous similar messages [ 5193.259696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.886274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.604470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.626826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.437152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.284803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.252557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.113411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.988429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.689473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.568898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.631249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.583224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.887466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.780146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.617394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.668122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.379978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.141811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.281776] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5211.293304] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5211.302102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.440149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.300597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.020152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.099421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.915720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.913331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.076433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.900861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.043658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.087480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.105471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.381451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.380957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.378300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.346859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.363542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.300688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.229190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.952140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.811274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.765063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.862288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.617860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.361488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.135338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.943242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.697699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.408701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.373812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.291433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.411655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.539735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.281869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.395862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.758625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.987692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.947068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.911376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.290338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.305778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.024510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.964145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.638814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.426125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.407753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.804511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.020588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.811415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.933699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.350621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.986932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.752655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.527069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.446835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.165554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.163786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.152727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.838417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.079753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.132038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.390334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.218900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.251899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.179758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.153358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.841744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.794597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.605722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.545801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.203586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.151821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.026883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.399136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.304617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.156883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.999712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.896077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.884723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.762661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.963876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.197879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.068894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.907383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.724413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.438864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.162931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.112571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.962819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.925237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.648856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.446321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.133062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.134569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.037727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.572951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.177790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.135962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.870242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.683481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.354668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.217799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.115327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.051545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.647339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.310580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.454534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.091606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.213966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.125435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.950370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.652308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.554940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.381311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.359722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.071500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.075940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.912424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.872893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.751520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.757987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.558695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.635202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.352504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.309176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.955237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.533228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.383411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.162372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.930820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.038404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.911603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.085467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.943207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.742553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.364425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.269551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.235151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.134963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.939244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.714502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.664442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.556447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.361694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.087271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.704844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.540723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.423086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.552756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.218661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.515450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.151647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.255919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.264708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.409060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.080484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.257368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.973450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.673703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.572844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.407807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.621751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.579782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.612156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.619227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.741082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.519155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.555639] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5366.571626] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5366.588579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.652879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.492836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.201462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.013714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.726085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.845712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.726247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.705159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.508973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.665426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.543296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.262167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.125368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.751290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.439960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.087371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.198795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.880179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.787910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.544651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.623858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.551991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.488929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.148771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.847630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.738211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.756938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.591859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.590300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.517801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.682594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.492649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.404721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.381321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.074736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.983843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.719817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.311385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.097153] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5400.103251] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5400.129285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.859196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.783089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.735702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.152950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.806610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.616809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.203598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.017865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.788306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.593611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.886402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.736634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.528463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.233867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.070457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.897647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.832375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.645856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.692974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.347807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.228805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.025727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.060266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.786024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.604484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.586698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.393830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.502964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.235539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.083422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.777797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.893668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.728946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.522949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.561590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.344083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.004652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.072190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.804668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.566616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.318888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.305892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.180946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.337086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.189290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.931388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.719375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.709354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.723379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.382400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.291418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.086962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.036502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.068239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.009117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.753258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.677599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.417784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.682672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.262315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.279603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.164083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.214326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.955722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.661528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.476282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.464679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.291725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.987321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.641056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.706466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.573028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.521061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.880883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.741528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.401273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.130673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.853566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.591059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.445730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.461838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.232141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.287431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.128663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.105893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.887427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.788755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.559969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.416851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.035849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.713084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.612280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.328182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.104031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.647271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.397556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.988842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.992294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.725312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.328081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.055251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.963094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.806250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.661547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.443317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.098848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.838423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.813951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.697348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.527454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.386656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.451965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.660676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.644298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.611564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.471916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.456705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.454282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.389679] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5502.400791] LustreError: 171938:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5502.415150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.393419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.203960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.062286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.756612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.106895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.318854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.085433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.901611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.800614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.692081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.390800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.114708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.745867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.760722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.507646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.295705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.062291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.984342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.994373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.691179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.713720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.491115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.309677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.337942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.189661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.147639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.250532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.934217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.785657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.458825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.549751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.267814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.393459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.308567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.136782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.038249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.060962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.270851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.887789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.999631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.712664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.933872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.823974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.912355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.727148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.986580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.908158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.522312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.270816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.196305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.122171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.345874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.488702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.332738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.123311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.176369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.101993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.176723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.350452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.486569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.143436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.218381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.051606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.144494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.835221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.614834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.660938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.448756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.301754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.555384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.506345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.693907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.507778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.289298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.014634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.984797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.055049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.887469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.805133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.653441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.460747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.285319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.130308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.946405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.857026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.772385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.751939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.608772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.782794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.713946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.475947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.965643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.070617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.206819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.120290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.180372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.108646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.148958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.156672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.125434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.864583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.635217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.240643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.252675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.843922] LustreError: 173740:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5903.851923] LustreError: 171938:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6319.084129] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6319.099518] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6319.105342] Lustre: Skipped 450 previous similar messages [ 6319.124645] Lustre: Skipped 1 previous similar message [ 6324.198502] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6324.215301] Lustre: Skipped 1 previous similar message [ 6324.344627] Lustre: server umount lustre-OST0000 complete [ 6329.315885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6329.320645] Lustre: Skipped 1 previous similar message [ 6333.817844] Lustre: server umount lustre-MDT0000 complete [ 6334.434893] LustreError: 171914:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6334.480947] LustreError: 171914:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6336.494114] LustreError: 171898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786437885 with bad export cookie 12393734369913387810 [ 6336.497706] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6336.511603] LustreError: 171898:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6336.792449] Lustre: server umount lustre-MDT0001 complete [ 6347.314501] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 6349.690361] Key type lgssc unregistered [ 6350.124896] LNet: 227118:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6350.142789] LNetError: 227118:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6350.222875] LNet: Removed LNI 192.168.203.109@tcp [ 6351.017464] Key type .llcrypt unregistered [ 6351.021489] Key type ._llcrypt unregistered [ 6373.568463] Key type ._llcrypt registered [ 6373.570647] Key type .llcrypt registered [ 6373.658731] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 6375.509855] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6375.522960] alg: No test for adler32 (adler32-zlib) [ 6376.569971] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 6376.763547] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 6378.455172] Key type lgssc registered [ 6379.215758] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6386.905979] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6386.918680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6388.295093] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6392.029875] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6398.437309] LustreError: 228515:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6398.453830] LustreError: 228515:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6399.048695] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6399.285208] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6403.298524] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6410.932493] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6416.100637] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6422.231340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6422.451398] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6427.473760] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6433.433336] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6433.807863] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6439.121133] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6441.059522] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6441.073486] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6441.081562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.092927] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6441.095844] LustreError: 228534:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6444.001695] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6444.004774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6444.048307] Lustre: Skipped 1 previous similar message [ 6444.061450] Lustre: Skipped 1 previous similar message [ 6446.907931] Lustre: lustre-OST0000: Not available for connect from 192.168.203.9@tcp (stopping) [ 6448.653650] Lustre: server umount lustre-OST0000 complete [ 6461.111135] LustreError: 228534:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6481.963351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6482.378262] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6484.388844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.417269] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.418902] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6484.481935] Lustre: Skipped 1 previous similar message [ 6488.658899] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6495.522618] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6497.010805] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6502.391904] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e08c33ad000.ost_server_uuid 50 [ 6504.212840] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e08c33ad000.ost_server_uuid in FULL state after 0 sec [ 6509.536985] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6509.546982] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6509.566968] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6513.577845] Lustre: server umount lustre-OST0000 complete [ 6520.295756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6520.299839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6520.313766] Lustre: Skipped 2 previous similar messages [ 6520.321851] Lustre: Skipped 3 previous similar messages [ 6523.713732] Lustre: server umount lustre-MDT0000 complete [ 6525.411965] LustreError: 228514:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6525.431480] LustreError: 228514:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6527.525771] LustreError: 228496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438076 with bad export cookie 5175394104053184222 [ 6527.530168] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6527.544399] LustreError: 228496:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6527.754650] Lustre: server umount lustre-MDT0001 complete [ 6539.221408] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 6541.660398] Key type lgssc unregistered [ 6541.949436] LNet: 233723:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6541.962861] LNetError: 233723:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6541.981686] LNet: Removed LNI 192.168.203.109@tcp [ 6542.927345] Key type .llcrypt unregistered [ 6542.933786] Key type ._llcrypt unregistered [ 6572.042909] Key type ._llcrypt registered [ 6572.047405] Key type .llcrypt registered [ 6572.187481] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 6573.415198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6573.569161] alg: No test for adler32 (adler32-zlib) [ 6574.848956] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 6575.041538] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 6576.727190] Key type lgssc registered [ 6578.093691] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6586.099621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6586.123652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6587.573259] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6591.264166] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6597.111489] LustreError: 235431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6597.145303] LustreError: 235431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6599.208430] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6599.486360] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6604.123479] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6606.672830] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6612.881293] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6613.119964] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6618.613754] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6626.679384] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6626.864297] Lustre: lustre-OST0001: new disk, initializing [ 6626.867711] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6626.875787] Lustre: 237736:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6627.014531] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6632.135354] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6633.298374] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6634.040138] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6634.051504] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6634.150654] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6644.385485] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6651.450261] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6664.660341] Lustre: server umount lustre-MDT0000 complete [ 6665.199749] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6665.206253] LustreError: 236835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6665.219136] Lustre: Skipped 3 previous similar messages [ 6665.239585] LustreError: 236835:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6668.068913] LustreError: 235412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438217 with bad export cookie 14052018088588310245 [ 6668.069462] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6668.082513] LustreError: 235412:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6670.339351] Lustre: server umount lustre-MDT0001 complete [ 6683.039696] Lustre: server umount lustre-OST0000 complete [ 6696.667192] Lustre: server umount lustre-OST0001 complete [ 6715.250039] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 6724.667050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6724.985985] LustreError: 240852:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6725.101632] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6729.655166] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6730.211630] LustreError: 240852:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6730.242179] LustreError: 240852:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6735.357706] LustreError: 240853:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6737.284313] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6737.643684] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6742.099771] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6745.401510] Lustre: 241994:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6752.273780] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6759.506223] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6761.969812] LustreError: 242348:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6768.005261] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6768.270065] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6768.279201] Lustre: Skipped 1 previous similar message [ 6770.292139] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 6774.101217] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6781.205690] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6784.184659] Lustre: 243865:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6796.262871] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6796.265541] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6796.291230] Lustre: Skipped 3 previous similar messages [ 6796.320144] Lustre: Skipped 3 previous similar messages [ 6800.777565] Lustre: server umount mylustre-MDT0000 complete [ 6801.500747] LustreError: 241584:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6801.588894] LustreError: 241584:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 6805.414846] LustreError: 243175:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438354 with bad export cookie 14052018088588313052 [ 6805.437103] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6805.447993] LustreError: 243175:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6805.851307] Lustre: server umount mylustre-MDT0001 complete [ 6819.167549] Lustre: server umount mylustre-OST0000 complete [ 6821.856388] Lustre: 234695:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786438355/real 1786438355] req@ffff997423229880 x1873216142481024/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786438371 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6821.895605] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6824.139333] Lustre: server umount mylustre-OST0001 complete [ 6843.719474] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 6854.327173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6854.591218] LustreError: 246173:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6854.655984] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6858.425531] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6867.099953] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6871.348719] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6874.206684] Lustre: 247315:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6881.544472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6889.200410] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6897.831441] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6899.146698] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 6904.241224] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 6912.391232] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6915.998894] Lustre: 249188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6923.751922] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6923.782076] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6923.850507] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6926.228060] Lustre: server umount tfs-MDT0000 complete [ 6929.908083] LustreError: 248954:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6929.949584] LustreError: 248954:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 6930.587887] LustreError: 249190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438479 with bad export cookie 14052018088588314718 [ 6930.595047] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6930.610463] LustreError: 249190:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 6930.836568] Lustre: server umount tfs-MDT0001 complete [ 6944.413951] Lustre: server umount tfs-OST0000 complete [ 6958.143412] Lustre: server umount tfs-OST0001 complete [ 6980.846385] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 6991.946293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6992.416870] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6992.424875] Lustre: Skipped 3 previous similar messages [ 6996.915487] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7005.413467] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7010.154936] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7012.896177] Lustre: 252583:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7019.725881] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7026.805475] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7035.928147] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7037.581868] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7044.082399] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7053.063405] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7056.954223] Lustre: 254457:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7066.738447] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 04:56:53 (1786438613) [ 7083.501417] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7083.512981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7083.525709] Lustre: Skipped 4 previous similar messages [ 7083.544671] Lustre: Skipped 7 previous similar messages [ 7088.661434] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7088.670737] Lustre: Skipped 3 previous similar messages [ 7089.377619] Lustre: server umount lustre-MDT0000 complete [ 7093.134140] LustreError: 251417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438642 with bad export cookie 14052018088588316160 [ 7093.136719] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7093.153191] LustreError: 251417:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7093.733208] LustreError: 254824:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7093.733939] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7093.735697] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7093.757910] LustreError: 254824:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 7093.772278] Lustre: Skipped 3 previous similar messages [ 7099.629804] Lustre: server umount lustre-MDT0001 complete [ 7109.592900] Lustre: server umount lustre-OST0000 complete [ 7128.031231] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7128.189981] Lustre: server umount lustre-OST0001 complete [ 7135.056959] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 7143.856559] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 7178.585390] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7178.585390] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7178.585390] [ 7178.614251] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7178.809895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7178.858372] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7178.936973] Lustre: lustre-MDT0000: new disk, initializing [ 7179.006927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7179.012709] Lustre: Skipped 3 previous similar messages [ 7179.028867] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7184.539279] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7195.590048] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7195.590048] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7195.590048] [ 7195.606919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7195.686872] Lustre: 257640:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7195.762828] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7195.767403] Lustre: Skipped 1 previous similar message [ 7195.858902] Lustre: lustre-MDT0001: new disk, initializing [ 7196.234499] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7196.249771] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7200.622479] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7211.261628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7211.657703] Lustre: lustre-OST0000: new disk, initializing [ 7211.662446] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7211.672411] Lustre: 259220:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7213.662307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7213.673520] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7213.751434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7217.501120] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7228.813433] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7228.999527] Lustre: lustre-OST0001: new disk, initializing [ 7229.003332] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7229.008862] Lustre: 260092:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7230.827113] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7230.841558] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7230.917265] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7234.559314] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7245.756257] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 04:59:53 (1786438793) [ 7258.435274] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:00:06 (1786438806) [ 7275.490434] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7275.494585] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7275.513823] LustreError: Skipped 1 previous similar message [ 7275.519975] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7275.519986] Lustre: Skipped 3 previous similar messages [ 7275.567701] Lustre: Skipped 1 previous similar message [ 7287.264446] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7287.537806] Lustre: server umount lustre-OST0000 complete [ 7292.398123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7292.406122] Lustre: Skipped 9 previous similar messages [ 7297.611794] Lustre: server umount lustre-MDT0000 complete [ 7301.448410] LustreError: 257631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786438850 with bad export cookie 14052018088588317490 [ 7301.450257] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7301.457265] LustreError: 257631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7308.023874] Lustre: server umount lustre-MDT0001 complete [ 7321.308981] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 7397.418589] Lustre: server umount lustre-OST0001 complete [ 7404.051989] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 7419.363929] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 7453.552885] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7453.818397] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7453.864429] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7453.938109] Lustre: lustre-MDT0000: new disk, initializing [ 7454.018050] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7454.022973] Lustre: Skipped 3 previous similar messages [ 7454.037694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7458.864685] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7470.551993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7470.667748] Lustre: 264903:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7470.734085] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7470.739078] Lustre: Skipped 1 previous similar message [ 7470.825581] Lustre: lustre-MDT0001: new disk, initializing [ 7471.320768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7471.336614] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7477.516496] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7487.577979] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7492.752389] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7498.817475] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7499.056525] Lustre: 266873:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7500.884033] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7501.052931] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7506.032815] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7515.241813] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7521.765400] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7521.767267] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7521.775453] LustreError: Skipped 1 previous similar message [ 7521.788175] Lustre: Skipped 4 previous similar messages [ 7521.802541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7521.809028] Lustre: Skipped 3 previous similar messages [ 7527.279377] Lustre: server umount lustre-OST0000 complete [ 7545.311120] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7545.494514] Lustre: server umount lustre-MDT0000 complete [ 7547.367851] LustreError: 265707:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7547.384582] LustreError: 265707:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [ 7548.378408] LustreError: 264896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786439097 with bad export cookie 14052018088588318953 [ 7548.383302] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7548.391555] LustreError: 264896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7548.614936] Lustre: server umount lustre-MDT0001 complete [ 7557.608628] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 7560.198928] Key type lgssc unregistered [ 7560.422753] LNet: 268637:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7560.444329] LNetError: 268637:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7561.509441] LNet: Removed LNI 192.168.203.109@tcp [ 7562.694249] Key type .llcrypt unregistered [ 7562.696937] Key type ._llcrypt unregistered [ 7589.172284] Key type ._llcrypt registered [ 7589.177147] Key type .llcrypt registered [ 7589.272323] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 7590.820808] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7590.891753] alg: No test for adler32 (adler32-zlib) [ 7591.905690] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 7592.102516] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 7593.743272] Key type lgssc registered [ 7594.828222] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7603.600234] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7603.615476] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7604.905932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7609.237890] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7615.457054] LustreError: 270100:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7615.571892] LustreError: 270100:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7616.492699] LustreError: 270099:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7618.338639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7618.741993] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7623.928727] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7632.435597] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7638.279558] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7644.684178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7644.988883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7650.947897] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7651.183705] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7657.926472] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7659.020371] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7661.541616] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7661.543054] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7661.560680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7663.597753] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7668.705888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7668.714669] Lustre: Skipped 1 previous similar message [ 7673.824379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7673.874345] Lustre: Skipped 1 previous similar message [ 7675.359724] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7675.467827] Lustre: server umount lustre-OST0000 complete [ 7678.947359] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7678.952758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7678.967939] Lustre: Skipped 1 previous similar message [ 7678.968218] Lustre: Skipped 1 previous similar message [ 7689.186055] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7689.203360] Lustre: Skipped 4 previous similar messages [ 7693.279163] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7693.482328] Lustre: server umount lustre-MDT0000 complete [ 7694.304369] LustreError: 270100:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7694.315579] LustreError: 270100:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7696.757456] LustreError: 270082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786439246 with bad export cookie 3244159504663680727 [ 7696.759251] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7696.770226] LustreError: 270082:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7697.061770] Lustre: server umount lustre-MDT0001 complete [ 7706.448585] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 7708.694708] Key type lgssc unregistered [ 7708.921246] LNet: 273696:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7708.927669] LNetError: 273696:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7708.948909] LNet: Removed LNI 192.168.203.109@tcp [ 7709.660932] Key type .llcrypt unregistered [ 7709.662581] Key type ._llcrypt unregistered [ 7725.764815] Key type ._llcrypt registered [ 7725.767292] Key type .llcrypt registered [ 7725.887772] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 7726.801418] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7726.813373] alg: No test for adler32 (adler32-zlib) [ 7727.776935] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 7727.963679] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 7729.607183] Key type lgssc registered [ 7730.424461] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7739.452090] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7739.491841] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7755.039414] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7760.161051] LustreError: 274890:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.109@tcp: failed processing log, type 4: rc = -110 [ 7786.911198] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7792.792311] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 7800.306324] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7802.301718] Lustre: server umount lustre-OST0000 complete [ 7819.285381] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 7821.432229] Key type lgssc unregistered [ 7821.644162] LNet: 276526:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7821.661960] LNetError: 276526:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7821.674993] LNet: Removed LNI 192.168.203.109@tcp [ 7822.325850] Key type .llcrypt unregistered [ 7822.328062] Key type ._llcrypt unregistered [ 7861.686498] Key type ._llcrypt registered [ 7861.689178] Key type .llcrypt registered [ 7861.763133] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 7863.461870] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7863.628440] alg: No test for adler32 (adler32-zlib) [ 7864.788403] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 7865.031501] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 7866.863184] Key type lgssc registered [ 7867.784753] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7880.684678] loop0: detected capacity change from 0 to 204800000 [ 7880.869688] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7880.911877] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7881.182967] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7882.043743] loop0: detected capacity change from 0 to 204800000 [ 7882.226271] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7882.245360] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7882.531506] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7883.727697] loop0: detected capacity change from 0 to 204800000 [ 7883.886467] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7883.900345] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7884.013560] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7884.950194] loop0: detected capacity change from 0 to 204800000 [ 7885.324647] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7887.088237] loop0: detected capacity change from 0 to 204800000 [ 7887.151724] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7890.850606] loop0: detected capacity change from 0 to 204800000 [ 7890.886669] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7894.880217] loop0: detected capacity change from 0 to 204800000 [ 7894.950784] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7898.721118] loop0: detected capacity change from 0 to 204800000 [ 7898.764383] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7902.913664] loop0: detected capacity change from 0 to 204800000 [ 7903.020663] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7903.124869] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7903.225114] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7903.233237] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7903.242755] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7903.252911] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7903.263559] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7903.272654] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7903.282771] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7903.292513] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7903.305880] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7903.316547] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7903.327413] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7903.335793] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7903.345992] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7903.357652] Lustre: 279940:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7908.795270] Lustre: server umount MGS complete [ 7909.702782] loop0: detected capacity change from 0 to 204800000 [ 7909.760692] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7910.140216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7910.211271] Lustre: lustre-MDT0000: Aborting client recovery [ 7911.087944] loop1: detected capacity change from 0 to 204800000 [ 7911.275298] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7911.337169] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7911.344985] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7911.352295] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7911.361449] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7911.369424] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7911.379093] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7911.405945] Lustre: 280372:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7911.442269] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7911.625103] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7911.684207] Lustre: lustre-MDT0001: Aborting client recovery [ 7912.486612] loop2: detected capacity change from 0 to 204800000 [ 7912.659644] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7912.912381] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7912.928725] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7912.948683] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7912.957433] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7912.969713] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7912.978646] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7912.987361] Lustre: 280452:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7913.105784] Lustre: Found index 0 for lustre-OST0000, updating log [ 7913.133706] Lustre: 280470:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7913.197453] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7913.229812] Lustre: lustre-OST0000: Aborting recovery [ 7914.125500] loop3: detected capacity change from 0 to 204800000 [ 7914.391826] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7914.471103] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7914.481790] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7914.492656] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7914.501710] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7914.509463] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7914.517512] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7914.525472] Lustre: 280535:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7914.539342] Lustre: Found index 1 for lustre-OST0001, updating log [ 7914.559839] Lustre: 280541:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7914.617213] Lustre: lustre-OST0001: Aborting recovery [ 7920.114125] LustreError: 280316:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 7921.138755] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7921.142076] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7921.148898] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7921.161993] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7921.251994] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7921.257371] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7923.770308] LustreError: 280283:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 786us: rc = -2 [ 7924.274417] LustreError: 280282:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 461us: rc = -2 [ 7924.287846] LustreError: 280282:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 16 previous similar messages [ 7937.505390] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7937.519662] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7937.537147] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7941.600559] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7941.601470] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7941.603985] Lustre: Skipped 1 previous similar message [ 7941.613284] Lustre: Skipped 1 previous similar message [ 7942.848836] Lustre: server umount lustre-MDT0000 complete [ 7943.851310] LustreError: 280266:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786439493 with bad export cookie 16509622954189832541 [ 7943.864619] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7946.720269] LustreError: 280281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7946.721456] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7946.721890] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7946.721895] Lustre: Skipped 2 previous similar messages [ 7946.734434] LustreError: 280281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7946.756379] Lustre: Skipped 2 previous similar messages [ 7948.768232] LustreError: 280282:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7948.769774] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7948.777850] LustreError: 280282:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7948.784574] Lustre: Skipped 2 previous similar messages [ 7950.305670] Lustre: server umount lustre-MDT0001 complete [ 7966.176503] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7966.618785] Lustre: server umount lustre-OST0000 complete [ 7973.709475] Lustre: server umount lustre-OST0001 complete [ 7982.142823] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 05:12:09 (1786439529) [ 8012.382339] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 8023.259440] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 8058.512439] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8058.725522] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8058.754127] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8058.803606] Lustre: lustre-MDT0000: new disk, initializing [ 8058.921581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8058.926661] Lustre: Skipped 1 previous similar message [ 8058.985689] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8064.027752] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8074.175723] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8074.263612] Lustre: 284617:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8074.309916] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8074.313428] Lustre: Skipped 1 previous similar message [ 8074.382600] Lustre: lustre-MDT0001: new disk, initializing [ 8074.493611] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8074.645260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8074.691555] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8079.177231] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8088.642482] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8094.296466] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8102.374634] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8102.729438] Lustre: lustre-OST0000: new disk, initializing [ 8102.738135] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8102.750798] Lustre: 286586:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8102.867992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8109.259918] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8112.261686] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8112.271632] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8112.355184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8118.353627] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8124.691680] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8136.383464] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8142.999129] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8151.751799] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8194.532816] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8194.542506] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8194.548515] LustreError: Skipped 1 previous similar message [ 8194.550556] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8194.581744] Lustre: Skipped 1 previous similar message [ 8204.782464] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8204.791035] Lustre: Skipped 3 previous similar messages [ 8208.351091] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8208.472304] Lustre: server umount lustre-OST0000 complete [ 8213.475366] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8213.482498] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8218.621104] Lustre: server umount lustre-MDT0000 complete [ 8220.199946] LustreError: 284622:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8220.249741] LustreError: 284622:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8222.832746] LustreError: 285422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786439772 with bad export cookie 16509622954189835243 [ 8222.835474] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8222.841408] LustreError: 285422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 8223.126465] Lustre: server umount lustre-MDT0001 complete [ 8232.362566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8237.739523] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8241.677614] Lustre: server umount MGS complete [ 8254.593461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8255.017028] LustreError: 290461:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8255.036472] LustreError: 290474:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8255.081965] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8259.372269] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8260.584191] LustreError: 290462:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8267.274209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8271.285534] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8277.944223] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8283.186445] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8290.910919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8291.389746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8291.393493] Lustre: Skipped 1 previous similar message [ 8297.989834] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8305.473901] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8333.284970] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8333.290139] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8333.296868] Lustre: Skipped 1 previous similar message [ 8333.301729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8333.305072] Lustre: Skipped 3 previous similar messages [ 8334.065720] Lustre: server umount lustre-OST0000 complete [ 8339.507655] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8347.159972] Lustre: server umount lustre-MDT0000 complete [ 8350.954928] LustreError: 293652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786439900 with bad export cookie 16509622954189837196 [ 8350.955464] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8350.967821] LustreError: 293652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8351.293823] Lustre: server umount lustre-MDT0001 complete [ 8362.875353] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 8365.736710] Key type lgssc unregistered [ 8365.932862] LNet: 294407:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8365.937745] LNetError: 294407:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8366.963755] LNet: Removed LNI 192.168.203.109@tcp [ 8368.022943] Key type .llcrypt unregistered [ 8368.025441] Key type ._llcrypt unregistered [ 8405.885567] Key type ._llcrypt registered [ 8405.887426] Key type .llcrypt registered [ 8405.990199] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 8421.939928] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 8423.780787] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8423.987511] alg: No test for adler32 (adler32-zlib) [ 8425.046385] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 8425.456238] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 8427.127160] Key type lgssc registered [ 8428.694622] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8463.150294] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8463.169794] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8464.395930] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8464.439942] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8464.508340] Lustre: lustre-MDT0000: new disk, initializing [ 8464.561890] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8464.575282] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8468.492046] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8478.459462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8478.593736] Lustre: 298366:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8478.636448] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8478.643814] Lustre: Skipped 1 previous similar message [ 8478.694174] Lustre: lustre-MDT0001: new disk, initializing [ 8478.771393] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8478.809395] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8478.833865] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8483.639322] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8491.339651] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8497.343189] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8504.476536] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8504.837313] Lustre: lustre-OST0000: new disk, initializing [ 8504.840826] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8504.848984] Lustre: 300332:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8504.974659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8511.468655] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8512.803430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8512.810429] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8513.039314] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8520.936758] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8528.366176] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8539.684178] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8547.751846] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8584.674326] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8584.674975] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8584.711469] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8588.640837] Lustre: server umount lustre-OST0000 complete [ 8594.925061] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8594.979879] Lustre: Skipped 1 previous similar message [ 8594.991628] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8594.999978] Lustre: Skipped 2 previous similar messages [ 8598.358935] Lustre: server umount lustre-MDT0000 complete [ 8601.997434] LustreError: 301831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786440151 with bad export cookie 17558820189113093415 [ 8602.003341] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8602.028076] LustreError: 301831:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8602.262691] Lustre: server umount lustre-MDT0001 complete [ 8610.949031] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8615.700636] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8620.828816] Lustre: server umount MGS complete [ 8632.103480] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8632.357944] LustreError: 304245:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8632.379768] LustreError: 304245:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8632.382476] LustreError: 304258:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8632.446558] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8636.320644] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8637.921478] LustreError: 304246:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8642.023553] LustreError: 304245:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8644.714138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8644.983823] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8649.213670] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8656.878813] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8662.402665] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8668.840319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8669.093254] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8675.376753] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8682.675441] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8711.152920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8711.156946] LustreError: Skipped 1 previous similar message [ 8711.158965] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8711.169730] Lustre: Skipped 1 previous similar message [ 8711.177432] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8713.186365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8713.190570] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8716.406746] Lustre: server umount lustre-OST0000 complete [ 8721.888304] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8721.892092] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8721.899654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8721.903679] Lustre: Skipped 1 previous similar message [ 8733.664888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8733.668953] Lustre: Skipped 5 previous similar messages [ 8734.175848] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8734.310133] Lustre: server umount lustre-MDT0000 complete [ 8737.701291] LustreError: 307083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786440286 with bad export cookie 17558820189113095242 [ 8737.710357] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8737.715610] LustreError: 307083:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8738.051800] Lustre: server umount lustre-MDT0001 complete [ 8748.409193] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [ 8750.837275] Key type lgssc unregistered [ 8751.124166] LNet: 308195:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8751.138703] LNetError: 308195:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8751.157699] LNet: Removed LNI 192.168.203.109@tcp [ 8751.754587] Key type .llcrypt unregistered [ 8751.757800] Key type ._llcrypt unregistered [ 8776.601515] Key type ._llcrypt registered [ 8776.603939] Key type .llcrypt registered [ 8776.679793] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 8792.344903] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 8794.015027] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8794.042243] alg: No test for adler32 (adler32-zlib) [ 8795.162716] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [ 8795.418564] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [ 8797.143185] Key type lgssc registered [ 8798.128406] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8820.423788] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 8834.006983] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8834.026458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8835.192432] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8835.252367] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8835.302245] Lustre: lustre-MDT0000: new disk, initializing [ 8835.349521] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8835.364096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8838.979975] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 8848.731505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8849.130481] Lustre: lustre-OST0000: new disk, initializing [ 8849.132973] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8849.136593] Lustre: Skipped 1 previous similar message [ 8849.144322] Lustre: 312246:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8849.215878] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8852.127645] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8852.150535] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8852.198434] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8855.675749] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9713.129811] Lustre: server umount lustre-MDT0000 complete [ 9717.114712] LustreError: 312245:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786441266 with bad export cookie 14057707573393365387 [ 9717.168463] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9727.651448] Lustre: server umount lustre-OST0000 complete [ 9752.930703] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [ 9763.421440] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [ 9795.609711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9795.851111] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9795.882945] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9795.972958] Lustre: lustre-MDT0000: new disk, initializing [ 9796.059463] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9796.075319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9800.393093] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9809.915844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9810.020456] Lustre: 316586:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9810.077455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9810.081883] Lustre: Skipped 1 previous similar message [ 9810.262648] Lustre: lustre-MDT0001: new disk, initializing [ 9810.347273] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9810.414422] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9810.442262] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9814.487035] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9824.983840] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9830.993731] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9838.318330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9838.638101] Lustre: lustre-OST0000: new disk, initializing [ 9838.641369] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9838.647552] Lustre: 318554:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9838.738674] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9844.143479] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9847.378092] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9847.383710] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9847.475453] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9853.649871] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9857.617212] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9857.800142] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9861.648880] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9861.823683] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9868.260635] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9868.261648] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9868.266611] LustreError: Skipped 1 previous similar message [ 9868.282083] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9868.290448] Lustre: Skipped 1 previous similar message [ 9870.065811] Lustre: server umount lustre-OST0000 complete [ 9876.961180] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9877.015497] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9877.029966] Lustre: Skipped 1 previous similar message [ 9877.041363] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9878.500504] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9878.501583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9878.524234] Lustre: Skipped 1 previous similar message [ 9879.799852] Lustre: server umount lustre-MDT0000 complete [ 9883.472141] LustreError: 316577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786441432 with bad export cookie 14057707573396306206 [ 9883.477755] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9883.481462] LustreError: 316577:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9883.742581] Lustre: server umount lustre-MDT0001 complete [ 9887.272802] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9888.798536] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 05:43:56 (1786441436) [ 9897.107562] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9897.514567] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9902.316116] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9902.568858] LustreError: 320867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9902.626955] LustreError: 320867:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9907.689316] LustreError: 320868:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9910.334044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9910.660267] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9915.026805] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9921.081606] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9926.919463] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9933.393329] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9933.616698] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9938.590974] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9944.615452] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9951.078818] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9951.204377] Lustre: lustre-OST0001: new disk, initializing [ 9951.208254] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9951.214687] Lustre: 323702:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9951.334692] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9957.919218] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [ 9960.564635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9960.572094] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9960.688579] LustreError: 324165:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9961.763790] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9961.772371] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9962.788566] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9962.834830] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9964.841971] LustreError: 324165:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9964.854379] LustreError: 324165:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9967.207049] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9967.427671] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9969.007690] LustreError: 324165:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9969.028452] LustreError: 324165:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9972.528466] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e08c7b22000.ost_server_uuid 50 [ 9973.742366] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e08c7b22000.ost_server_uuid in FULL state after 0 sec [ 9977.254239] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9977.268710] LustreError: 324174:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [ 9978.783715] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8e08c7b22000.ost_server_uuid 50 [ 9980.500760] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8e08c7b22000.ost_server_uuid in FULL state after 0 sec [ 9982.460506] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9996.774167] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9996.781149] Lustre: Skipped 1 previous similar message [ 9996.784445] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10001.655293] Lustre: server umount lustre-OST0001 complete [10001.905653] LustreError: 322688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10001.938055] LustreError: 322688:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10007.031436] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10007.034053] LustreError: 324166:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10007.056091] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10007.056100] Lustre: Skipped 2 previous similar messages [10007.066943] Lustre: Skipped 1 previous similar message [10007.109061] LustreError: 324166:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10012.151558] LustreError: 322691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10012.190222] LustreError: 322691:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10012.629985] Lustre: server umount lustre-OST0000 complete [10017.248957] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10017.251243] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10017.265497] Lustre: Skipped 1 previous similar message [10017.266782] Lustre: Skipped 3 previous similar messages [10022.320418] Lustre: server umount lustre-MDT0000 complete [10022.368963] LustreError: 320863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10022.393625] LustreError: 320863:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10025.763455] LustreError: 323701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786441575 with bad export cookie 14057707573396307088 [10025.768228] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10025.774731] LustreError: 323701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10026.062891] Lustre: server umount lustre-MDT0001 complete [10036.788875] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [10039.361428] Key type lgssc unregistered [10039.764777] LNet: 326391:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10039.769800] LNetError: 326391:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10039.780109] LNet: Removed LNI 192.168.203.109@tcp [10040.609934] Key type .llcrypt unregistered [10040.611590] Key type ._llcrypt unregistered [10069.082633] Key type ._llcrypt registered [10069.084820] Key type .llcrypt registered [10069.192717] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10070.552650] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10070.574547] alg: No test for adler32 (adler32-zlib) [10071.650666] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [10071.862210] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [10073.567695] Key type lgssc registered [10074.384473] Lustre: Echo OBD driver; http://www.lustre.org/ [10081.037972] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10081.056654] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10082.363929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10086.175854] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10092.512436] LustreError: 327818:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10092.522811] LustreError: 327818:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10094.128709] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10097.643343] LustreError: 327817:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10101.018233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10101.244635] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10105.244573] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10114.707183] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10115.030481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10120.809905] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10126.330371] LustreError: 329498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10127.348471] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10128.680965] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10132.460638] LustreError: 329498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10132.479556] LustreError: 329498:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10135.344703] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10135.514476] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10140.673961] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10141.987890] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10150.190243] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10156.723285] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10157.007960] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10258.406123] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10258.414790] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10258.440291] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10261.488509] Lustre: server umount lustre-OST0001 complete [10263.566863] LustreError: 330089:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10263.585114] LustreError: 330089:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10268.647090] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10268.652329] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10268.661254] Lustre: Skipped 2 previous similar messages [10268.666737] Lustre: Skipped 1 previous similar message [10272.427696] Lustre: server umount lustre-OST0000 complete [10278.880644] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10278.894593] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10278.911887] Lustre: Skipped 1 previous similar message [10278.934618] Lustre: Skipped 2 previous similar messages [10283.626472] Lustre: server umount lustre-MDT0000 complete [10284.007315] LustreError: 327812:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10284.049313] LustreError: 327812:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10287.189059] LustreError: 331500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786441836 with bad export cookie 8896997812325706097 [10287.191772] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10287.207425] LustreError: 331500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10287.503590] Lustre: server umount lustre-MDT0001 complete [10298.855214] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [10300.971566] Key type lgssc unregistered [10301.310936] LNet: 332763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10301.314708] LNetError: 332763:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10301.324430] LNet: Removed LNI 192.168.203.109@tcp [10301.895619] Key type .llcrypt unregistered [10301.897728] Key type ._llcrypt unregistered [10346.315077] Key type ._llcrypt registered [10346.317212] Key type .llcrypt registered [10346.433311] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10348.008126] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10348.053428] alg: No test for adler32 (adler32-zlib) [10349.132700] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [10349.359850] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [10350.999270] Key type lgssc registered [10352.256216] Lustre: Echo OBD driver; http://www.lustre.org/ [10360.368741] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10360.383360] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10361.691621] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10365.466666] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10371.047448] LustreError: 335051:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10371.059074] LustreError: 335051:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10373.401878] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10373.844499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10378.399652] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10381.373541] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10386.949641] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10387.477388] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10395.420311] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10398.768391] LustreError: 336541:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10403.818163] LustreError: 336543:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10405.309051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10405.525240] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10406.531457] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10406.533907] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10411.564363] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10422.734449] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10431.236723] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10443.747743] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10443.752133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10443.762282] Lustre: Skipped 2 previous similar messages [10443.766284] Lustre: Skipped 1 previous similar message [10448.179978] Lustre: server umount lustre-MDT0000 complete [10448.875831] LustreError: 335052:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10448.926525] LustreError: 335052:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10451.609500] LustreError: 335032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786442000 with bad export cookie 15234142556574555104 [10451.614163] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10451.620454] LustreError: 335032:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10451.862568] Lustre: server umount lustre-MDT0001 complete [10465.690176] Lustre: server umount lustre-OST0000 complete [10479.760432] Lustre: server umount lustre-OST0001 complete [10495.428807] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10505.262464] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10506.090145] LustreError: 340318:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10506.233678] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10510.775883] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10516.479830] LustreError: 340319:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10516.513981] LustreError: 340319:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10520.978239] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10521.628869] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10526.251846] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10529.176950] Lustre: 341457:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10535.612878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10542.572130] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10546.216626] LustreError: 341811:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10546.239317] LustreError: 341811:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10551.981105] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10552.210826] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10552.222555] Lustre: Skipped 1 previous similar message [10553.299497] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10553.340100] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10557.667633] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10566.006748] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10569.282749] Lustre: 343331:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10573.792711] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10573.800046] Lustre: Skipped 1 previous similar message [10573.802790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10573.805615] Lustre: Skipped 2 previous similar messages [10578.917380] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10578.932792] Lustre: Skipped 4 previous similar messages [10579.337498] Lustre: server umount lustre-MDT0000 complete [10582.997502] LustreError: 340300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786442132 with bad export cookie 15234142556574556805 [10582.999801] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10583.019228] LustreError: 340300:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10583.370385] Lustre: server umount lustre-MDT0001 complete [10595.988841] Lustre: server umount lustre-OST0000 complete [10609.885660] Lustre: server umount lustre-OST0001 complete [10613.204639] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10615.312491] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10617.441736] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:56:04 (1786442164) [10639.586577] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10647.821972] loop0: detected capacity change from 0 to 1099511627776 [10648.319633] SGI XFS with ACLs, security attributes, quota, no debug enabled [10648.324564] *** VALIDATE xfs *** [10648.331685] XFS (loop0): Mounting V5 Filesystem [10648.347310] XFS (loop0): Ending clean mount [10649.275521] loop1: detected capacity change from 0 to 18691697672192 [10703.963535] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [10713.236314] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10750.079384] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10750.539783] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10750.560051] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10750.614355] Lustre: lustre-MDT0000: new disk, initializing [10750.666739] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10750.687662] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10755.307200] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10765.679202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10765.848329] Lustre: 348465:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10765.903695] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10765.911785] Lustre: Skipped 1 previous similar message [10765.964942] Lustre: lustre-MDT0001: new disk, initializing [10766.103581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10766.112620] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10771.286926] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10781.470246] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10787.577284] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10794.965322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10795.262189] Lustre: lustre-OST0000: new disk, initializing [10795.265688] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10795.271649] Lustre: 350432:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10796.954141] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10796.977870] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10797.143543] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10801.615522] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10810.547572] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10815.864766] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10816.179255] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10820.217782] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10820.420688] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10822.626700] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10822.627893] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10822.640665] Lustre: Skipped 2 previous similar messages [10822.643276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10822.651169] LustreError: Skipped 1 previous similar message [10822.665122] Lustre: Skipped 1 previous similar message [10827.750325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10827.754041] Lustre: Skipped 1 previous similar message [10828.421807] Lustre: server umount lustre-OST0000 complete [10832.864557] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10832.865510] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10832.871351] Lustre: Skipped 1 previous similar message [10837.911152] Lustre: server umount lustre-MDT0000 complete [10837.985596] LustreError: 348475:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10838.071986] LustreError: 348475:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10841.326376] LustreError: 348457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786442390 with bad export cookie 15234142556574558352 [10841.334741] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10841.341487] LustreError: 348457:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10841.615710] Lustre: server umount lustre-MDT0001 complete [10850.357902] XFS (loop0): Unmounting Filesystem [10854.778133] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 06:00:02 (1786442402) [10863.705489] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10864.298831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10864.305475] Lustre: Skipped 2 previous similar messages [10868.995948] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10877.234683] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10881.537454] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10888.301996] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10893.946220] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10900.482646] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10906.485772] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [10913.761671] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10923.488633] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10923.496256] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10923.507920] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10923.513020] Lustre: Skipped 2 previous similar messages [10927.485707] Lustre: server umount lustre-OST0000 complete [10933.728485] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10933.734983] LustreError: Skipped 1 previous similar message [10933.742543] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10933.757601] Lustre: Skipped 1 previous similar message [10937.119417] Lustre: server umount lustre-MDT0000 complete [10939.873835] LustreError: 352986:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10939.893536] LustreError: 352986:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [10940.283554] LustreError: 352970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786442489 with bad export cookie 15234142556574559227 [10940.292118] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10940.293944] LustreError: 352970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10940.490738] Lustre: server umount lustre-MDT0001 complete [10950.689855] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [10953.094804] Key type lgssc unregistered [10953.309591] LNet: 356630:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10953.315820] LNetError: 356630:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10953.328656] LNet: Removed LNI 192.168.203.109@tcp [10954.021161] Key type .llcrypt unregistered [10954.023202] Key type ._llcrypt unregistered [10983.039504] Key type ._llcrypt registered [10983.041385] Key type .llcrypt registered [10983.209292] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [10984.967580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10984.983834] alg: No test for adler32 (adler32-zlib) [10986.201093] Lustre: Lustre: Build Version: 2.17.56_51_gd63417b [10986.364200] LNet: Added LNI 192.168.203.109@tcp [8/256/0/180] [10988.055428] Key type lgssc registered [10989.211735] Lustre: Echo OBD driver; http://www.lustre.org/ [10995.802440] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10995.815614] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10997.133476] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11001.399622] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11007.456542] LustreError: 358200:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11007.480473] LustreError: 358200:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11009.669254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11010.038714] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11014.274530] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11021.485933] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11027.140108] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11033.076527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11033.372927] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11038.471520] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11045.625827] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11052.807434] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.9@tcp (stopping) [11053.089046] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11057.907466] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.9@tcp (stopping) [11057.912062] Lustre: Skipped 2 previous similar messages [11058.432660] Lustre: server umount lustre-MDT0001 complete [11063.042614] LustreError: 358195:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.9@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.134843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11068.138886] LustreError: 358196:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.9@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11068.168248] LustreError: 358196:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11068.214910] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11068.219228] Lustre: Found index 1 for lustre-MDT0001, updating log [11068.242387] Lustre: 358188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11068.480459] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11073.523093] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11074.087864] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11076.591064] LustreError: 361780:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11076.597292] LustreError: 361780:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11076.607699] LustreError: 358193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11076.646814] LustreError: 361789:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.109@tcp: cfg command failed: rc = -17 [11076.655719] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11076.655719] [11083.583480] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.9@tcp (stopping) [11083.592528] Lustre: Skipped 2 previous similar messages [11083.750428] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11083.754702] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11083.783235] Lustre: Skipped 2 previous similar messages [11088.670748] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.9@tcp (stopping) [11088.679829] Lustre: Skipped 2 previous similar messages [11088.985974] LustreError: 358194:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11089.231508] Lustre: server umount lustre-MDT0001 complete [11093.732947] LustreError: 358195:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.9@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11093.752054] LustreError: 358195:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11099.265557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11099.393101] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11099.397869] Lustre: Found index 1 for lustre-MDT0001, updating log [11099.427601] Lustre: 358189:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11099.752806] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11104.009500] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11104.749493] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11104.790360] LustreError: 362757:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11104.797525] LustreError: 362757:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11104.827420] LustreError: 358193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11104.856101] LustreError: 358193:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11105.161797] LustreError: 362777:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11105.167308] LustreError: 362777:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11105.171424] LustreError: 362777:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.109@tcp: cfg command failed: rc = -17 [11105.189294] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.109@tcp [11105.189294] [11411.197286] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.9@tcp (stopping) [11412.451770] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11412.454636] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11415.889848] Lustre: server umount lustre-MDT0001 complete [11416.326771] LustreError: 358194:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.9@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11416.367195] LustreError: 358194:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11426.612491] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11426.684845] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11426.689331] Lustre: Found index 1 for lustre-MDT0001, updating log [11426.723786] Lustre: 358188:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11426.981663] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11431.956646] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11432.426731] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11433.985775] LustreError: 372024:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11434.005694] LustreError: 372024:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11434.021703] LustreError: 358193:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11434.031296] LustreError: 358193:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11434.094306] LustreError: 372032:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11434.119270] LustreError: 372032:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.109@tcp: cfg command failed: rc = -114 [11434.138197] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11434.138197] [11754.977666] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11754.982369] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11754.982751] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11754.982756] Lustre: Skipped 2 previous similar messages [11754.985022] Lustre: Skipped 2 previous similar messages [11765.218237] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11765.370502] Lustre: server umount lustre-MDT0000 complete [11768.447050] LustreError: 358182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786443317 with bad export cookie 3615152885813156946 [11768.449898] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11768.455668] LustreError: 358182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11770.337251] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11770.337778] LustreError: 358194:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11770.344652] Lustre: Skipped 1 previous similar message [11770.398347] LustreError: 358194:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [11773.346673] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11773.353105] Lustre: Skipped 9 previous similar messages [11774.796301] Lustre: server umount lustre-MDT0001 complete [11778.054434] Lustre: server umount lustre-OST0000 complete [11785.803721] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [11793.484725] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [11821.458192] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:16:09 (1786443369) [11827.647350] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11827.833974] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11827.858396] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11827.923943] Lustre: lustre-MDT0000: new disk, initializing [11827.991519] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11828.010381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11832.690745] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11842.733618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11842.807398] Lustre: 383429:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11842.849445] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11842.852972] Lustre: Skipped 1 previous similar message [11842.918374] Lustre: lustre-MDT0001: new disk, initializing [11843.003687] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11843.064094] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11843.083400] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11846.627615] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11854.394245] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11859.572238] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11865.376798] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11865.576541] Lustre: lustre-OST0000: new disk, initializing [11865.579419] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11865.585345] Lustre: 385396:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11865.655804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11870.590557] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_default_debug -1 all [11875.430705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11875.446297] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11875.633260] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11878.460599] Lustre: DEBUG MARKER: oleg309-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11886.050388] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11886.053395] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11886.069572] Lustre: Skipped 2 previous similar messages [11886.086425] Lustre: Skipped 2 previous similar messages [11890.528801] Lustre: server umount lustre-MDT0000 complete [11891.177056] LustreError: 383434:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11891.195579] LustreError: 383434:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11893.492320] LustreError: 383420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786443442 with bad export cookie 3615152885813158906 [11893.501274] LustreError: MGC192.168.203.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11893.504174] LustreError: 384232:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11893.517569] LustreError: 384232:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@0000000056691631 x1873220767875456/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [11893.729259] Lustre: server umount lustre-MDT0001 complete [11907.124769] Lustre: server umount lustre-OST0000 complete [11934.035265] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing set_hostid [11942.592366] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing load_modules_local [11971.279504] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11662 sec ========= 06:18:39 (1786443519) [11973.284480] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:18:40 (1786443520) === [11976.124405] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:18:43 (1786443523) === [12012.464613] Lustre: DEBUG MARKER: oleg309-server.virtnet: executing unload_modules_local [12014.864372] Key type lgssc unregistered [12015.166811] LNet: 391763:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12015.171829] LNetError: 391763:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12016.229855] LNet: Removed LNI 192.168.203.109@tcp [12016.869689] Key type .llcrypt unregistered [12016.872376] Key type ._llcrypt unregistered